2023-02-13T02:02:53,000 Created temporary directory: /tmp/pip-ephem-wheel-cache-2_denapq 2023-02-13T02:02:53,005 Created temporary directory: /tmp/pip-build-tracker-m8sjly61 2023-02-13T02:02:53,006 Initialized build tracking at /tmp/pip-build-tracker-m8sjly61 2023-02-13T02:02:53,006 Created build tracker: /tmp/pip-build-tracker-m8sjly61 2023-02-13T02:02:53,006 Entered build tracker: /tmp/pip-build-tracker-m8sjly61 2023-02-13T02:02:53,008 Created temporary directory: /tmp/pip-wheel-za_z2e0n 2023-02-13T02:02:53,016 DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453 2023-02-13T02:02:53,021 Created temporary directory: /tmp/pip-ephem-wheel-cache-k7hlq7wg 2023-02-13T02:02:53,072 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2023-02-13T02:02:53,080 2 location(s) to search for versions of colossalai: 2023-02-13T02:02:53,080 * https://pypi.org/simple/colossalai/ 2023-02-13T02:02:53,080 * https://www.piwheels.org/simple/colossalai/ 2023-02-13T02:02:53,081 Fetching project page and analyzing links: https://pypi.org/simple/colossalai/ 2023-02-13T02:02:53,082 Getting page https://pypi.org/simple/colossalai/ 2023-02-13T02:02:53,085 Found index url https://pypi.org/simple 2023-02-13T02:02:53,319 Fetched page https://pypi.org/simple/colossalai/ as application/vnd.pypi.simple.v1+json 2023-02-13T02:02:53,324 Found link https://files.pythonhosted.org/packages/9c/c7/427dd2ed3769c6beb05cefdc18f8fad634673ed1472d5409afbdb768c7c3/colossalai-0.2.0.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.0 2023-02-13T02:02:53,326 Found link https://files.pythonhosted.org/packages/b1/70/cc6165ba3ce239c4b0c53c8ea6cabbfb31aa4b66dc580e85ff16d787384d/colossalai-0.2.1.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.1 2023-02-13T02:02:53,327 Found link https://files.pythonhosted.org/packages/87/d0/d82b49b722315654f6773eb458b929dba9949044a6b7cd087027c2391169/colossalai-0.2.2.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.2 2023-02-13T02:02:53,328 Found link https://files.pythonhosted.org/packages/c9/bb/9ec93ed622207b589fe4b1eddad9da74174eedc40ac306bff77ba823db92/colossalai-0.2.3.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6), version: 0.2.3 2023-02-13T02:02:53,329 Fetching project page and analyzing links: https://www.piwheels.org/simple/colossalai/ 2023-02-13T02:02:53,330 Getting page https://www.piwheels.org/simple/colossalai/ 2023-02-13T02:02:53,332 Found index url https://www.piwheels.org/simple 2023-02-13T02:02:53,550 Fetched page https://www.piwheels.org/simple/colossalai/ as text/html 2023-02-13T02:02:53,555 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.2-py3-none-any.whl#sha256=272ef339dfcdc089fc4e150a9ed2377e0998b1b3e5061b8760add23228609122 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-02-13T02:02:53,555 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.1-py3-none-any.whl#sha256=fba9ebaec6b5f43d7e2424a858cae18f765cf95d36956496e4476f23f36d97b0 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-02-13T02:02:53,556 Skipping link: No binaries permitted for colossalai: https://www.piwheels.org/simple/colossalai/colossalai-0.2.0-py3-none-any.whl#sha256=707a78c4c8d4b96171adc704e95bbdc387fc1dee4f612a8500f0a562c4454dc7 (from https://www.piwheels.org/simple/colossalai/) (requires-python:>=3.6) 2023-02-13T02:02:53,557 Skipping link: not a file: https://www.piwheels.org/simple/colossalai/ 2023-02-13T02:02:53,557 Skipping link: not a file: https://pypi.org/simple/colossalai/ 2023-02-13T02:02:53,591 Given no hashes to check 1 links for project 'colossalai': discarding no candidates 2023-02-13T02:02:53,621 Collecting colossalai==0.2.3 2023-02-13T02:02:53,626 Created temporary directory: /tmp/pip-unpack-h69font2 2023-02-13T02:02:54,295 Downloading colossalai-0.2.3.tar.gz (705 kB) 2023-02-13T02:02:57,589 Added colossalai==0.2.3 from https://files.pythonhosted.org/packages/c9/bb/9ec93ed622207b589fe4b1eddad9da74174eedc40ac306bff77ba823db92/colossalai-0.2.3.tar.gz to build tracker '/tmp/pip-build-tracker-m8sjly61' 2023-02-13T02:02:57,592 Running setup.py (path:/tmp/pip-wheel-za_z2e0n/colossalai_e2e5a858d6414fd1b66c3f72b68de6c1/setup.py) egg_info for package colossalai 2023-02-13T02:02:57,593 Created temporary directory: /tmp/pip-pip-egg-info-vr0qx_lf 2023-02-13T02:02:57,594 Preparing metadata (setup.py): started 2023-02-13T02:02:57,596 Running command python setup.py egg_info 2023-02-13T02:03:00,446 running egg_info 2023-02-13T02:03:00,451 creating /tmp/pip-pip-egg-info-vr0qx_lf/colossalai.egg-info 2023-02-13T02:03:00,516 writing /tmp/pip-pip-egg-info-vr0qx_lf/colossalai.egg-info/PKG-INFO 2023-02-13T02:03:00,522 writing dependency_links to /tmp/pip-pip-egg-info-vr0qx_lf/colossalai.egg-info/dependency_links.txt 2023-02-13T02:03:00,525 writing entry points to /tmp/pip-pip-egg-info-vr0qx_lf/colossalai.egg-info/entry_points.txt 2023-02-13T02:03:00,528 writing requirements to /tmp/pip-pip-egg-info-vr0qx_lf/colossalai.egg-info/requires.txt 2023-02-13T02:03:00,531 writing top-level names to /tmp/pip-pip-egg-info-vr0qx_lf/colossalai.egg-info/top_level.txt 2023-02-13T02:03:00,533 writing manifest file '/tmp/pip-pip-egg-info-vr0qx_lf/colossalai.egg-info/SOURCES.txt' 2023-02-13T02:03:00,939 reading manifest file '/tmp/pip-pip-egg-info-vr0qx_lf/colossalai.egg-info/SOURCES.txt' 2023-02-13T02:03:00,942 reading manifest template 'MANIFEST.in' 2023-02-13T02:03:03,104 warning: no files found matching '*.tr' under directory 'colossalai' 2023-02-13T02:03:04,182 warning: no files found matching '*.cc' under directory 'colossalai' 2023-02-13T02:03:04,702 warning: no files found matching '*.pyi' under directory 'colossalai' 2023-02-13T02:03:04,726 adding license file 'LICENSE' 2023-02-13T02:03:04,793 writing manifest file '/tmp/pip-pip-egg-info-vr0qx_lf/colossalai.egg-info/SOURCES.txt' 2023-02-13T02:03:04,999 Preparing metadata (setup.py): finished with status 'done' 2023-02-13T02:03:05,015 Source in /tmp/pip-wheel-za_z2e0n/colossalai_e2e5a858d6414fd1b66c3f72b68de6c1 has version 0.2.3, which satisfies requirement colossalai==0.2.3 from https://files.pythonhosted.org/packages/c9/bb/9ec93ed622207b589fe4b1eddad9da74174eedc40ac306bff77ba823db92/colossalai-0.2.3.tar.gz 2023-02-13T02:03:05,017 Removed colossalai==0.2.3 from https://files.pythonhosted.org/packages/c9/bb/9ec93ed622207b589fe4b1eddad9da74174eedc40ac306bff77ba823db92/colossalai-0.2.3.tar.gz from build tracker '/tmp/pip-build-tracker-m8sjly61' 2023-02-13T02:03:05,036 Created temporary directory: /tmp/pip-unpack-hpz7vqew 2023-02-13T02:03:05,037 Building wheels for collected packages: colossalai 2023-02-13T02:03:05,046 Created temporary directory: /tmp/pip-wheel-g7izrfet 2023-02-13T02:03:05,047 Building wheel for colossalai (setup.py): started 2023-02-13T02:03:05,049 Destination directory: /tmp/pip-wheel-g7izrfet 2023-02-13T02:03:05,049 Running command python setup.py bdist_wheel 2023-02-13T02:03:07,700 running bdist_wheel 2023-02-13T02:03:08,474 running build 2023-02-13T02:03:08,475 running build_py 2023-02-13T02:03:08,546 creating build 2023-02-13T02:03:08,547 creating build/lib 2023-02-13T02:03:08,548 creating build/lib/op_builder 2023-02-13T02:03:08,550 copying op_builder/cpu_adam.py -> build/lib/op_builder 2023-02-13T02:03:08,554 copying op_builder/__init__.py -> build/lib/op_builder 2023-02-13T02:03:08,557 copying op_builder/scaled_masked_softmax.py -> build/lib/op_builder 2023-02-13T02:03:08,560 copying op_builder/moe.py -> build/lib/op_builder 2023-02-13T02:03:08,563 copying op_builder/scaled_upper_triangle_masked_softmax.py -> build/lib/op_builder 2023-02-13T02:03:08,566 copying op_builder/builder.py -> build/lib/op_builder 2023-02-13T02:03:08,569 copying op_builder/multi_head_attn.py -> build/lib/op_builder 2023-02-13T02:03:08,572 copying op_builder/utils.py -> build/lib/op_builder 2023-02-13T02:03:08,575 copying op_builder/layernorm.py -> build/lib/op_builder 2023-02-13T02:03:08,578 copying op_builder/fused_optim.py -> build/lib/op_builder 2023-02-13T02:03:08,583 creating build/lib/colossalai 2023-02-13T02:03:08,584 copying colossalai/__init__.py -> build/lib/colossalai 2023-02-13T02:03:08,587 copying colossalai/constants.py -> build/lib/colossalai 2023-02-13T02:03:08,591 copying colossalai/core.py -> build/lib/colossalai 2023-02-13T02:03:08,594 copying colossalai/initialize.py -> build/lib/colossalai 2023-02-13T02:03:08,599 copying colossalai/version.py -> build/lib/colossalai 2023-02-13T02:03:08,602 copying colossalai/global_variables.py -> build/lib/colossalai 2023-02-13T02:03:08,607 creating build/lib/colossalai/engine 2023-02-13T02:03:08,609 copying colossalai/engine/__init__.py -> build/lib/colossalai/engine 2023-02-13T02:03:08,613 copying colossalai/engine/_base_engine.py -> build/lib/colossalai/engine 2023-02-13T02:03:08,619 creating build/lib/colossalai/context 2023-02-13T02:03:08,621 copying colossalai/context/moe_context.py -> build/lib/colossalai/context 2023-02-13T02:03:08,626 copying colossalai/context/__init__.py -> build/lib/colossalai/context 2023-02-13T02:03:08,630 copying colossalai/context/config.py -> build/lib/colossalai/context 2023-02-13T02:03:08,634 copying colossalai/context/parallel_context.py -> build/lib/colossalai/context 2023-02-13T02:03:08,639 copying colossalai/context/parallel_mode.py -> build/lib/colossalai/context 2023-02-13T02:03:08,643 copying colossalai/context/singleton_meta.py -> build/lib/colossalai/context 2023-02-13T02:03:08,648 creating build/lib/colossalai/trainer 2023-02-13T02:03:08,650 copying colossalai/trainer/__init__.py -> build/lib/colossalai/trainer 2023-02-13T02:03:08,654 copying colossalai/trainer/_trainer.py -> build/lib/colossalai/trainer 2023-02-13T02:03:08,661 creating build/lib/colossalai/builder 2023-02-13T02:03:08,662 copying colossalai/builder/__init__.py -> build/lib/colossalai/builder 2023-02-13T02:03:08,665 copying colossalai/builder/builder.py -> build/lib/colossalai/builder 2023-02-13T02:03:08,669 creating build/lib/colossalai/_C 2023-02-13T02:03:08,671 copying colossalai/_C/__init__.py -> build/lib/colossalai/_C 2023-02-13T02:03:08,675 creating build/lib/colossalai/registry 2023-02-13T02:03:08,676 copying colossalai/registry/__init__.py -> build/lib/colossalai/registry 2023-02-13T02:03:08,680 copying colossalai/registry/registry.py -> build/lib/colossalai/registry 2023-02-13T02:03:08,684 creating build/lib/colossalai/logging 2023-02-13T02:03:08,685 copying colossalai/logging/logger.py -> build/lib/colossalai/logging 2023-02-13T02:03:08,689 copying colossalai/logging/__init__.py -> build/lib/colossalai/logging 2023-02-13T02:03:08,693 creating build/lib/colossalai/auto_parallel 2023-02-13T02:03:08,694 copying colossalai/auto_parallel/__init__.py -> build/lib/colossalai/auto_parallel 2023-02-13T02:03:08,699 creating build/lib/colossalai/gemini 2023-02-13T02:03:08,700 copying colossalai/gemini/stateful_tensor_mgr.py -> build/lib/colossalai/gemini 2023-02-13T02:03:08,704 copying colossalai/gemini/__init__.py -> build/lib/colossalai/gemini 2023-02-13T02:03:08,707 copying colossalai/gemini/gemini_mgr.py -> build/lib/colossalai/gemini 2023-02-13T02:03:08,711 copying colossalai/gemini/gemini_context.py -> build/lib/colossalai/gemini 2023-02-13T02:03:08,714 copying colossalai/gemini/placement_policy.py -> build/lib/colossalai/gemini 2023-02-13T02:03:08,718 copying colossalai/gemini/stateful_tensor.py -> build/lib/colossalai/gemini 2023-02-13T02:03:08,721 copying colossalai/gemini/tensor_utils.py -> build/lib/colossalai/gemini 2023-02-13T02:03:08,724 copying colossalai/gemini/tensor_placement_policy.py -> build/lib/colossalai/gemini 2023-02-13T02:03:08,729 creating build/lib/colossalai/nn 2023-02-13T02:03:08,730 copying colossalai/nn/__init__.py -> build/lib/colossalai/nn 2023-02-13T02:03:08,734 copying colossalai/nn/init.py -> build/lib/colossalai/nn 2023-02-13T02:03:08,739 creating build/lib/colossalai/zero 2023-02-13T02:03:08,740 copying colossalai/zero/__init__.py -> build/lib/colossalai/zero 2023-02-13T02:03:08,744 creating build/lib/colossalai/amp 2023-02-13T02:03:08,746 copying colossalai/amp/__init__.py -> build/lib/colossalai/amp 2023-02-13T02:03:08,749 copying colossalai/amp/amp_type.py -> build/lib/colossalai/amp 2023-02-13T02:03:08,754 creating build/lib/colossalai/utils 2023-02-13T02:03:08,755 copying colossalai/utils/__init__.py -> build/lib/colossalai/utils 2023-02-13T02:03:08,759 copying colossalai/utils/moe.py -> build/lib/colossalai/utils 2023-02-13T02:03:08,762 copying colossalai/utils/timer.py -> build/lib/colossalai/utils 2023-02-13T02:03:08,766 copying colossalai/utils/cuda.py -> build/lib/colossalai/utils 2023-02-13T02:03:08,769 copying colossalai/utils/common.py -> build/lib/colossalai/utils 2023-02-13T02:03:08,774 copying colossalai/utils/checkpointing.py -> build/lib/colossalai/utils 2023-02-13T02:03:08,778 copying colossalai/utils/memory.py -> build/lib/colossalai/utils 2023-02-13T02:03:08,782 copying colossalai/utils/activation_checkpoint.py -> build/lib/colossalai/utils 2023-02-13T02:03:08,787 creating build/lib/colossalai/communication 2023-02-13T02:03:08,788 copying colossalai/communication/__init__.py -> build/lib/colossalai/communication 2023-02-13T02:03:08,791 copying colossalai/communication/p2p_v2.py -> build/lib/colossalai/communication 2023-02-13T02:03:08,795 copying colossalai/communication/ring.py -> build/lib/colossalai/communication 2023-02-13T02:03:08,798 copying colossalai/communication/p2p.py -> build/lib/colossalai/communication 2023-02-13T02:03:08,802 copying colossalai/communication/collective.py -> build/lib/colossalai/communication 2023-02-13T02:03:08,806 copying colossalai/communication/utils.py -> build/lib/colossalai/communication 2023-02-13T02:03:08,810 creating build/lib/colossalai/kernel 2023-02-13T02:03:08,812 copying colossalai/kernel/__init__.py -> build/lib/colossalai/kernel 2023-02-13T02:03:08,816 creating build/lib/colossalai/device 2023-02-13T02:03:08,818 copying colossalai/device/__init__.py -> build/lib/colossalai/device 2023-02-13T02:03:08,821 copying colossalai/device/alpha_beta_profiler.py -> build/lib/colossalai/device 2023-02-13T02:03:08,825 copying colossalai/device/device_mesh.py -> build/lib/colossalai/device 2023-02-13T02:03:08,828 copying colossalai/device/calc_pipeline_strategy.py -> build/lib/colossalai/device 2023-02-13T02:03:08,833 creating build/lib/colossalai/fx 2023-02-13T02:03:08,834 copying colossalai/fx/graph_module.py -> build/lib/colossalai/fx 2023-02-13T02:03:08,838 copying colossalai/fx/__init__.py -> build/lib/colossalai/fx 2023-02-13T02:03:08,841 copying colossalai/fx/_meta_registrations.py -> build/lib/colossalai/fx 2023-02-13T02:03:08,845 copying colossalai/fx/_compatibility.py -> build/lib/colossalai/fx 2023-02-13T02:03:08,848 copying colossalai/fx/proxy.py -> build/lib/colossalai/fx 2023-02-13T02:03:08,853 creating build/lib/colossalai/testing 2023-02-13T02:03:08,854 copying colossalai/testing/__init__.py -> build/lib/colossalai/testing 2023-02-13T02:03:08,858 copying colossalai/testing/comparison.py -> build/lib/colossalai/testing 2023-02-13T02:03:08,861 copying colossalai/testing/utils.py -> build/lib/colossalai/testing 2023-02-13T02:03:08,865 copying colossalai/testing/random.py -> build/lib/colossalai/testing 2023-02-13T02:03:08,868 copying colossalai/testing/pytest_wrapper.py -> build/lib/colossalai/testing 2023-02-13T02:03:08,872 creating build/lib/colossalai/pipeline 2023-02-13T02:03:08,874 copying colossalai/pipeline/__init__.py -> build/lib/colossalai/pipeline 2023-02-13T02:03:08,878 copying colossalai/pipeline/utils.py -> build/lib/colossalai/pipeline 2023-02-13T02:03:08,882 copying colossalai/pipeline/pipelinable.py -> build/lib/colossalai/pipeline 2023-02-13T02:03:08,886 copying colossalai/pipeline/layer_spec.py -> build/lib/colossalai/pipeline 2023-02-13T02:03:08,890 copying colossalai/pipeline/pipeline_process_group.py -> build/lib/colossalai/pipeline 2023-02-13T02:03:08,894 creating build/lib/colossalai/cli 2023-02-13T02:03:08,896 copying colossalai/cli/__init__.py -> build/lib/colossalai/cli 2023-02-13T02:03:08,899 copying colossalai/cli/cli.py -> build/lib/colossalai/cli 2023-02-13T02:03:08,905 creating build/lib/colossalai/tensor 2023-02-13T02:03:08,906 copying colossalai/tensor/const.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,910 copying colossalai/tensor/dist_spec_mgr.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,913 copying colossalai/tensor/__init__.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,916 copying colossalai/tensor/sharding_spec.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,921 copying colossalai/tensor/colo_tensor.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,925 copying colossalai/tensor/distspec.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,928 copying colossalai/tensor/param_op_hook.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,932 copying colossalai/tensor/utils.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,936 copying colossalai/tensor/compute_spec.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,939 copying colossalai/tensor/tensor_spec.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,942 copying colossalai/tensor/comm_spec.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,947 copying colossalai/tensor/colo_parameter.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,951 copying colossalai/tensor/shape_consistency.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,956 copying colossalai/tensor/op_wrapper.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,960 copying colossalai/tensor/process_group.py -> build/lib/colossalai/tensor 2023-02-13T02:03:08,965 creating build/lib/colossalai/engine/schedule 2023-02-13T02:03:08,967 copying colossalai/engine/schedule/__init__.py -> build/lib/colossalai/engine/schedule 2023-02-13T02:03:08,970 copying colossalai/engine/schedule/_pipeline_schedule_v2.py -> build/lib/colossalai/engine/schedule 2023-02-13T02:03:08,974 copying colossalai/engine/schedule/_non_pipeline_schedule.py -> build/lib/colossalai/engine/schedule 2023-02-13T02:03:08,978 copying colossalai/engine/schedule/_base_schedule.py -> build/lib/colossalai/engine/schedule 2023-02-13T02:03:08,981 copying colossalai/engine/schedule/_pipeline_schedule.py -> build/lib/colossalai/engine/schedule 2023-02-13T02:03:08,989 creating build/lib/colossalai/engine/gradient_handler 2023-02-13T02:03:08,990 copying colossalai/engine/gradient_handler/__init__.py -> build/lib/colossalai/engine/gradient_handler 2023-02-13T02:03:08,994 copying colossalai/engine/gradient_handler/_data_parallel_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-02-13T02:03:08,997 copying colossalai/engine/gradient_handler/_pipeline_parallel_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-02-13T02:03:09,000 copying colossalai/engine/gradient_handler/_base_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-02-13T02:03:09,003 copying colossalai/engine/gradient_handler/_moe_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-02-13T02:03:09,007 copying colossalai/engine/gradient_handler/utils.py -> build/lib/colossalai/engine/gradient_handler 2023-02-13T02:03:09,010 copying colossalai/engine/gradient_handler/_sequence_parallel_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-02-13T02:03:09,013 copying colossalai/engine/gradient_handler/_zero_gradient_handler.py -> build/lib/colossalai/engine/gradient_handler 2023-02-13T02:03:09,017 creating build/lib/colossalai/engine/gradient_accumulation 2023-02-13T02:03:09,019 copying colossalai/engine/gradient_accumulation/__init__.py -> build/lib/colossalai/engine/gradient_accumulation 2023-02-13T02:03:09,022 copying colossalai/engine/gradient_accumulation/_gradient_accumulation.py -> build/lib/colossalai/engine/gradient_accumulation 2023-02-13T02:03:09,028 creating build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,030 copying colossalai/context/process_group_initializer/__init__.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,034 copying colossalai/context/process_group_initializer/initializer_3d.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,038 copying colossalai/context/process_group_initializer/process_group_initializer.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,041 copying colossalai/context/process_group_initializer/initializer_2p5d.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,045 copying colossalai/context/process_group_initializer/initializer_data.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,048 copying colossalai/context/process_group_initializer/initializer_2d.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,052 copying colossalai/context/process_group_initializer/initializer_sequence.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,056 copying colossalai/context/process_group_initializer/initializer_pipeline.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,060 copying colossalai/context/process_group_initializer/initializer_1d.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,063 copying colossalai/context/process_group_initializer/initializer_model.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,067 copying colossalai/context/process_group_initializer/initializer_tensor.py -> build/lib/colossalai/context/process_group_initializer 2023-02-13T02:03:09,072 creating build/lib/colossalai/context/random 2023-02-13T02:03:09,074 copying colossalai/context/random/__init__.py -> build/lib/colossalai/context/random 2023-02-13T02:03:09,078 copying colossalai/context/random/seed_manager.py -> build/lib/colossalai/context/random 2023-02-13T02:03:09,082 copying colossalai/context/random/_helper.py -> build/lib/colossalai/context/random 2023-02-13T02:03:09,088 creating build/lib/colossalai/trainer/hooks 2023-02-13T02:03:09,090 copying colossalai/trainer/hooks/__init__.py -> build/lib/colossalai/trainer/hooks 2023-02-13T02:03:09,094 copying colossalai/trainer/hooks/_metric_hook.py -> build/lib/colossalai/trainer/hooks 2023-02-13T02:03:09,098 copying colossalai/trainer/hooks/_lr_scheduler_hook.py -> build/lib/colossalai/trainer/hooks 2023-02-13T02:03:09,101 copying colossalai/trainer/hooks/_checkpoint_hook.py -> build/lib/colossalai/trainer/hooks 2023-02-13T02:03:09,105 copying colossalai/trainer/hooks/_commons_.py -> build/lib/colossalai/trainer/hooks 2023-02-13T02:03:09,109 copying colossalai/trainer/hooks/_base_hook.py -> build/lib/colossalai/trainer/hooks 2023-02-13T02:03:09,114 copying colossalai/trainer/hooks/_log_hook.py -> build/lib/colossalai/trainer/hooks 2023-02-13T02:03:09,120 creating build/lib/colossalai/auto_parallel/tensor_shard 2023-02-13T02:03:09,122 copying colossalai/auto_parallel/tensor_shard/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-02-13T02:03:09,125 copying colossalai/auto_parallel/tensor_shard/constants.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-02-13T02:03:09,129 copying colossalai/auto_parallel/tensor_shard/sharding_strategy.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-02-13T02:03:09,134 copying colossalai/auto_parallel/tensor_shard/initialize.py -> build/lib/colossalai/auto_parallel/tensor_shard 2023-02-13T02:03:09,140 creating build/lib/colossalai/auto_parallel/pipeline_shard 2023-02-13T02:03:09,142 copying colossalai/auto_parallel/pipeline_shard/__init__.py -> build/lib/colossalai/auto_parallel/pipeline_shard 2023-02-13T02:03:09,147 creating build/lib/colossalai/auto_parallel/meta_profiler 2023-02-13T02:03:09,149 copying colossalai/auto_parallel/meta_profiler/__init__.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-02-13T02:03:09,153 copying colossalai/auto_parallel/meta_profiler/registry.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-02-13T02:03:09,157 copying colossalai/auto_parallel/meta_profiler/constants.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-02-13T02:03:09,160 copying colossalai/auto_parallel/meta_profiler/metainfo.py -> build/lib/colossalai/auto_parallel/meta_profiler 2023-02-13T02:03:09,166 creating build/lib/colossalai/auto_parallel/passes 2023-02-13T02:03:09,168 copying colossalai/auto_parallel/passes/__init__.py -> build/lib/colossalai/auto_parallel/passes 2023-02-13T02:03:09,171 copying colossalai/auto_parallel/passes/constants.py -> build/lib/colossalai/auto_parallel/passes 2023-02-13T02:03:09,175 copying colossalai/auto_parallel/passes/runtime_apply_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-02-13T02:03:09,180 copying colossalai/auto_parallel/passes/meta_info_prop.py -> build/lib/colossalai/auto_parallel/passes 2023-02-13T02:03:09,184 copying colossalai/auto_parallel/passes/runtime_preparation_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-02-13T02:03:09,189 copying colossalai/auto_parallel/passes/comm_metainfo_pass.py -> build/lib/colossalai/auto_parallel/passes 2023-02-13T02:03:09,196 creating build/lib/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:09,198 copying colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:09,203 copying colossalai/auto_parallel/checkpoint/__init__.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:09,206 copying colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:09,209 copying colossalai/auto_parallel/checkpoint/build_c_ext.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:09,212 copying colossalai/auto_parallel/checkpoint/operation.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:09,215 copying colossalai/auto_parallel/checkpoint/ckpt_solver_base.py -> build/lib/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:09,221 creating build/lib/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:09,223 copying colossalai/auto_parallel/tensor_shard/deprecated/solver.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:09,228 copying colossalai/auto_parallel/tensor_shard/deprecated/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:09,231 copying colossalai/auto_parallel/tensor_shard/deprecated/constants.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:09,234 copying colossalai/auto_parallel/tensor_shard/deprecated/graph_analysis.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:09,238 copying colossalai/auto_parallel/tensor_shard/deprecated/sharding_strategy.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:09,242 copying colossalai/auto_parallel/tensor_shard/deprecated/cost_graph.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:09,246 copying colossalai/auto_parallel/tensor_shard/deprecated/strategies_constructor.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:09,251 copying colossalai/auto_parallel/tensor_shard/deprecated/options.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:09,254 copying colossalai/auto_parallel/tensor_shard/deprecated/_utils.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:09,259 creating build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:09,261 copying colossalai/auto_parallel/tensor_shard/solver/solver.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:09,265 copying colossalai/auto_parallel/tensor_shard/solver/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:09,268 copying colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:09,273 copying colossalai/auto_parallel/tensor_shard/solver/cost_graph.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:09,277 copying colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:09,280 copying colossalai/auto_parallel/tensor_shard/solver/options.py -> build/lib/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:09,288 creating build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,289 copying colossalai/auto_parallel/tensor_shard/node_handler/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,292 copying colossalai/auto_parallel/tensor_shard/node_handler/registry.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,296 copying colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,299 copying colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,303 copying colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,307 copying colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,310 copying colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,314 copying colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,318 copying colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,322 copying colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,326 copying colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,329 copying colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,333 copying colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,337 copying colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,342 copying colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,346 copying colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,351 copying colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,354 copying colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,358 copying colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,362 copying colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,365 copying colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,368 copying colossalai/auto_parallel/tensor_shard/node_handler/option.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,371 copying colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,374 copying colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,378 copying colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,381 copying colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,385 copying colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,389 copying colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:09,394 creating build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:09,396 copying colossalai/auto_parallel/tensor_shard/utils/misc.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:09,399 copying colossalai/auto_parallel/tensor_shard/utils/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:09,402 copying colossalai/auto_parallel/tensor_shard/utils/broadcast.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:09,406 copying colossalai/auto_parallel/tensor_shard/utils/reshape.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:09,410 copying colossalai/auto_parallel/tensor_shard/utils/factory.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:09,414 copying colossalai/auto_parallel/tensor_shard/utils/sharding.py -> build/lib/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:09,420 creating build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,422 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,426 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/operator_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,430 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/batch_norm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,436 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/unary_elementwise_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,440 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/reshape_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,443 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/embedding_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,447 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/where_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,451 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/layer_norm_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,456 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,459 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/bcast_op_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,465 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/dot_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,470 copying colossalai/auto_parallel/tensor_shard/deprecated/op_handler/conv_handler.py -> build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:09,478 creating build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,479 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,484 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,488 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,492 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,495 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,499 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,502 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,506 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,510 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,514 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,518 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,522 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,526 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,531 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,535 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,538 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,542 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,546 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,552 copying colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py -> build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:09,557 creating build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:09,558 copying colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:09,562 copying colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:09,565 copying colossalai/auto_parallel/meta_profiler/meta_registry/linear.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:09,570 copying colossalai/auto_parallel/meta_profiler/meta_registry/activation.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:09,573 copying colossalai/auto_parallel/meta_profiler/meta_registry/conv.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:09,577 copying colossalai/auto_parallel/meta_profiler/meta_registry/norm.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:09,581 copying colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py -> build/lib/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:09,586 creating build/lib/colossalai/gemini/paramhooks 2023-02-13T02:03:09,588 copying colossalai/gemini/paramhooks/__init__.py -> build/lib/colossalai/gemini/paramhooks 2023-02-13T02:03:09,591 copying colossalai/gemini/paramhooks/_param_hookmgr.py -> build/lib/colossalai/gemini/paramhooks 2023-02-13T02:03:09,596 creating build/lib/colossalai/gemini/memory_tracer 2023-02-13T02:03:09,598 copying colossalai/gemini/memory_tracer/__init__.py -> build/lib/colossalai/gemini/memory_tracer 2023-02-13T02:03:09,601 copying colossalai/gemini/memory_tracer/static_memstats_collector.py -> build/lib/colossalai/gemini/memory_tracer 2023-02-13T02:03:09,605 copying colossalai/gemini/memory_tracer/runtime_mem_tracer.py -> build/lib/colossalai/gemini/memory_tracer 2023-02-13T02:03:09,608 copying colossalai/gemini/memory_tracer/memory_monitor.py -> build/lib/colossalai/gemini/memory_tracer 2023-02-13T02:03:09,611 copying colossalai/gemini/memory_tracer/utils.py -> build/lib/colossalai/gemini/memory_tracer 2023-02-13T02:03:09,615 copying colossalai/gemini/memory_tracer/memstats_collector.py -> build/lib/colossalai/gemini/memory_tracer 2023-02-13T02:03:09,618 copying colossalai/gemini/memory_tracer/param_runtime_order.py -> build/lib/colossalai/gemini/memory_tracer 2023-02-13T02:03:09,621 copying colossalai/gemini/memory_tracer/memory_stats.py -> build/lib/colossalai/gemini/memory_tracer 2023-02-13T02:03:09,626 copying colossalai/gemini/memory_tracer/chunk_memstats_collector.py -> build/lib/colossalai/gemini/memory_tracer 2023-02-13T02:03:09,632 creating build/lib/colossalai/gemini/chunk 2023-02-13T02:03:09,634 copying colossalai/gemini/chunk/__init__.py -> build/lib/colossalai/gemini/chunk 2023-02-13T02:03:09,638 copying colossalai/gemini/chunk/chunk.py -> build/lib/colossalai/gemini/chunk 2023-02-13T02:03:09,643 copying colossalai/gemini/chunk/utils.py -> build/lib/colossalai/gemini/chunk 2023-02-13T02:03:09,646 copying colossalai/gemini/chunk/manager.py -> build/lib/colossalai/gemini/chunk 2023-02-13T02:03:09,650 copying colossalai/gemini/chunk/search_utils.py -> build/lib/colossalai/gemini/chunk 2023-02-13T02:03:09,655 creating build/lib/colossalai/gemini/ophooks 2023-02-13T02:03:09,657 copying colossalai/gemini/ophooks/__init__.py -> build/lib/colossalai/gemini/ophooks 2023-02-13T02:03:09,660 copying colossalai/gemini/ophooks/_shard_param_ophook.py -> build/lib/colossalai/gemini/ophooks 2023-02-13T02:03:09,663 copying colossalai/gemini/ophooks/_shard_grad_ophook.py -> build/lib/colossalai/gemini/ophooks 2023-02-13T02:03:09,667 copying colossalai/gemini/ophooks/utils.py -> build/lib/colossalai/gemini/ophooks 2023-02-13T02:03:09,671 copying colossalai/gemini/ophooks/runtime_mem_tracer_hook.py -> build/lib/colossalai/gemini/ophooks 2023-02-13T02:03:09,679 creating build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,681 copying colossalai/nn/_ops/loss.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,685 copying colossalai/nn/_ops/__init__.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,688 copying colossalai/nn/_ops/linear.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,692 copying colossalai/nn/_ops/embedding_bag.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,696 copying colossalai/nn/_ops/view.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,699 copying colossalai/nn/_ops/batch_norm.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,702 copying colossalai/nn/_ops/addmm.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,706 copying colossalai/nn/_ops/element_wise.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,710 copying colossalai/nn/_ops/layernorm.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,713 copying colossalai/nn/_ops/embedding.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,717 copying colossalai/nn/_ops/_utils.py -> build/lib/colossalai/nn/_ops 2023-02-13T02:03:09,723 creating build/lib/colossalai/nn/parallel 2023-02-13T02:03:09,725 copying colossalai/nn/parallel/gemini_parallel.py -> build/lib/colossalai/nn/parallel 2023-02-13T02:03:09,729 copying colossalai/nn/parallel/__init__.py -> build/lib/colossalai/nn/parallel 2023-02-13T02:03:09,732 copying colossalai/nn/parallel/utils.py -> build/lib/colossalai/nn/parallel 2023-02-13T02:03:09,736 copying colossalai/nn/parallel/data_parallel.py -> build/lib/colossalai/nn/parallel 2023-02-13T02:03:09,741 copying colossalai/nn/parallel/reducer.py -> build/lib/colossalai/nn/parallel 2023-02-13T02:03:09,745 copying colossalai/nn/parallel/zero_wrapper.py -> build/lib/colossalai/nn/parallel 2023-02-13T02:03:09,750 creating build/lib/colossalai/nn/loss 2023-02-13T02:03:09,751 copying colossalai/nn/loss/__init__.py -> build/lib/colossalai/nn/loss 2023-02-13T02:03:09,755 copying colossalai/nn/loss/loss_2d.py -> build/lib/colossalai/nn/loss 2023-02-13T02:03:09,759 copying colossalai/nn/loss/loss_moe.py -> build/lib/colossalai/nn/loss 2023-02-13T02:03:09,762 copying colossalai/nn/loss/loss_2p5d.py -> build/lib/colossalai/nn/loss 2023-02-13T02:03:09,766 copying colossalai/nn/loss/loss_3d.py -> build/lib/colossalai/nn/loss 2023-02-13T02:03:09,769 copying colossalai/nn/loss/loss_1d.py -> build/lib/colossalai/nn/loss 2023-02-13T02:03:09,775 creating build/lib/colossalai/nn/lr_scheduler 2023-02-13T02:03:09,776 copying colossalai/nn/lr_scheduler/cosine.py -> build/lib/colossalai/nn/lr_scheduler 2023-02-13T02:03:09,780 copying colossalai/nn/lr_scheduler/__init__.py -> build/lib/colossalai/nn/lr_scheduler 2023-02-13T02:03:09,784 copying colossalai/nn/lr_scheduler/linear.py -> build/lib/colossalai/nn/lr_scheduler 2023-02-13T02:03:09,787 copying colossalai/nn/lr_scheduler/onecycle.py -> build/lib/colossalai/nn/lr_scheduler 2023-02-13T02:03:09,791 copying colossalai/nn/lr_scheduler/poly.py -> build/lib/colossalai/nn/lr_scheduler 2023-02-13T02:03:09,794 copying colossalai/nn/lr_scheduler/torch.py -> build/lib/colossalai/nn/lr_scheduler 2023-02-13T02:03:09,797 copying colossalai/nn/lr_scheduler/delayed.py -> build/lib/colossalai/nn/lr_scheduler 2023-02-13T02:03:09,801 copying colossalai/nn/lr_scheduler/multistep.py -> build/lib/colossalai/nn/lr_scheduler 2023-02-13T02:03:09,807 creating build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,808 copying colossalai/nn/optimizer/hybrid_adam.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,812 copying colossalai/nn/optimizer/fused_adam.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,816 copying colossalai/nn/optimizer/cpu_adam.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,820 copying colossalai/nn/optimizer/lars.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,823 copying colossalai/nn/optimizer/__init__.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,826 copying colossalai/nn/optimizer/fused_lamb.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,830 copying colossalai/nn/optimizer/gemini_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,834 copying colossalai/nn/optimizer/fused_sgd.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,837 copying colossalai/nn/optimizer/colossalai_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,841 copying colossalai/nn/optimizer/zero_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,845 copying colossalai/nn/optimizer/nvme_optimizer.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,849 copying colossalai/nn/optimizer/lamb.py -> build/lib/colossalai/nn/optimizer 2023-02-13T02:03:09,854 creating build/lib/colossalai/nn/layer 2023-02-13T02:03:09,855 copying colossalai/nn/layer/__init__.py -> build/lib/colossalai/nn/layer 2023-02-13T02:03:09,859 copying colossalai/nn/layer/base_layer.py -> build/lib/colossalai/nn/layer 2023-02-13T02:03:09,864 creating build/lib/colossalai/nn/metric 2023-02-13T02:03:09,865 copying colossalai/nn/metric/accuracy_2p5d.py -> build/lib/colossalai/nn/metric 2023-02-13T02:03:09,868 copying colossalai/nn/metric/__init__.py -> build/lib/colossalai/nn/metric 2023-02-13T02:03:09,872 copying colossalai/nn/metric/accuracy_2d.py -> build/lib/colossalai/nn/metric 2023-02-13T02:03:09,875 copying colossalai/nn/metric/accuracy_3d.py -> build/lib/colossalai/nn/metric 2023-02-13T02:03:09,878 copying colossalai/nn/metric/_utils.py -> build/lib/colossalai/nn/metric 2023-02-13T02:03:09,883 creating build/lib/colossalai/nn/parallel/layers 2023-02-13T02:03:09,884 copying colossalai/nn/parallel/layers/__init__.py -> build/lib/colossalai/nn/parallel/layers 2023-02-13T02:03:09,888 copying colossalai/nn/parallel/layers/linear.py -> build/lib/colossalai/nn/parallel/layers 2023-02-13T02:03:09,891 copying colossalai/nn/parallel/layers/module_utils.py -> build/lib/colossalai/nn/parallel/layers 2023-02-13T02:03:09,895 copying colossalai/nn/parallel/layers/colo_module.py -> build/lib/colossalai/nn/parallel/layers 2023-02-13T02:03:09,899 copying colossalai/nn/parallel/layers/embedding.py -> build/lib/colossalai/nn/parallel/layers 2023-02-13T02:03:09,905 creating build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:09,906 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:09,911 copying colossalai/nn/parallel/layers/cache_embedding/__init__.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:09,915 copying colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:09,919 copying colossalai/nn/parallel/layers/cache_embedding/embedding_config.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:09,923 copying colossalai/nn/parallel/layers/cache_embedding/copyer.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:09,928 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:09,932 copying colossalai/nn/parallel/layers/cache_embedding/base_embedding.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:09,936 copying colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:09,941 copying colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py -> build/lib/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:09,948 creating build/lib/colossalai/nn/layer/parallel_2d 2023-02-13T02:03:09,950 copying colossalai/nn/layer/parallel_2d/layers.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-02-13T02:03:09,957 copying colossalai/nn/layer/parallel_2d/__init__.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-02-13T02:03:09,961 copying colossalai/nn/layer/parallel_2d/_operation.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-02-13T02:03:09,966 copying colossalai/nn/layer/parallel_2d/_utils.py -> build/lib/colossalai/nn/layer/parallel_2d 2023-02-13T02:03:09,972 creating build/lib/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:09,974 copying colossalai/nn/layer/colossalai_layer/__init__.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:09,978 copying colossalai/nn/layer/colossalai_layer/linear.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:09,982 copying colossalai/nn/layer/colossalai_layer/normalization.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:09,986 copying colossalai/nn/layer/colossalai_layer/dropout.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:09,990 copying colossalai/nn/layer/colossalai_layer/embedding.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:10,001 copying colossalai/nn/layer/colossalai_layer/_utils.py -> build/lib/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:10,007 creating build/lib/colossalai/nn/layer/parallel_2p5d 2023-02-13T02:03:10,009 copying colossalai/nn/layer/parallel_2p5d/layers.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-02-13T02:03:10,016 copying colossalai/nn/layer/parallel_2p5d/__init__.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-02-13T02:03:10,020 copying colossalai/nn/layer/parallel_2p5d/_operation.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-02-13T02:03:10,026 copying colossalai/nn/layer/parallel_2p5d/_utils.py -> build/lib/colossalai/nn/layer/parallel_2p5d 2023-02-13T02:03:10,031 creating build/lib/colossalai/nn/layer/utils 2023-02-13T02:03:10,033 copying colossalai/nn/layer/utils/__init__.py -> build/lib/colossalai/nn/layer/utils 2023-02-13T02:03:10,037 copying colossalai/nn/layer/utils/common.py -> build/lib/colossalai/nn/layer/utils 2023-02-13T02:03:10,042 creating build/lib/colossalai/nn/layer/parallel_3d 2023-02-13T02:03:10,044 copying colossalai/nn/layer/parallel_3d/layers.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-02-13T02:03:10,051 copying colossalai/nn/layer/parallel_3d/__init__.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-02-13T02:03:10,054 copying colossalai/nn/layer/parallel_3d/_operation.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-02-13T02:03:10,059 copying colossalai/nn/layer/parallel_3d/_utils.py -> build/lib/colossalai/nn/layer/parallel_3d 2023-02-13T02:03:10,064 creating build/lib/colossalai/nn/layer/moe 2023-02-13T02:03:10,065 copying colossalai/nn/layer/moe/layers.py -> build/lib/colossalai/nn/layer/moe 2023-02-13T02:03:10,069 copying colossalai/nn/layer/moe/__init__.py -> build/lib/colossalai/nn/layer/moe 2023-02-13T02:03:10,072 copying colossalai/nn/layer/moe/experts.py -> build/lib/colossalai/nn/layer/moe 2023-02-13T02:03:10,076 copying colossalai/nn/layer/moe/utils.py -> build/lib/colossalai/nn/layer/moe 2023-02-13T02:03:10,079 copying colossalai/nn/layer/moe/routers.py -> build/lib/colossalai/nn/layer/moe 2023-02-13T02:03:10,083 copying colossalai/nn/layer/moe/_operation.py -> build/lib/colossalai/nn/layer/moe 2023-02-13T02:03:10,088 creating build/lib/colossalai/nn/layer/parallel_sequence 2023-02-13T02:03:10,089 copying colossalai/nn/layer/parallel_sequence/layers.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-02-13T02:03:10,094 copying colossalai/nn/layer/parallel_sequence/__init__.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-02-13T02:03:10,097 copying colossalai/nn/layer/parallel_sequence/_operation.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-02-13T02:03:10,100 copying colossalai/nn/layer/parallel_sequence/_utils.py -> build/lib/colossalai/nn/layer/parallel_sequence 2023-02-13T02:03:10,105 creating build/lib/colossalai/nn/layer/parallel_1d 2023-02-13T02:03:10,106 copying colossalai/nn/layer/parallel_1d/layers.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-02-13T02:03:10,112 copying colossalai/nn/layer/parallel_1d/__init__.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-02-13T02:03:10,115 copying colossalai/nn/layer/parallel_1d/_operation.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-02-13T02:03:10,118 copying colossalai/nn/layer/parallel_1d/_utils.py -> build/lib/colossalai/nn/layer/parallel_1d 2023-02-13T02:03:10,123 creating build/lib/colossalai/nn/layer/wrapper 2023-02-13T02:03:10,125 copying colossalai/nn/layer/wrapper/__init__.py -> build/lib/colossalai/nn/layer/wrapper 2023-02-13T02:03:10,128 copying colossalai/nn/layer/wrapper/pipeline_wrapper.py -> build/lib/colossalai/nn/layer/wrapper 2023-02-13T02:03:10,132 creating build/lib/colossalai/nn/layer/vanilla 2023-02-13T02:03:10,133 copying colossalai/nn/layer/vanilla/layers.py -> build/lib/colossalai/nn/layer/vanilla 2023-02-13T02:03:10,138 copying colossalai/nn/layer/vanilla/__init__.py -> build/lib/colossalai/nn/layer/vanilla 2023-02-13T02:03:10,142 creating build/lib/colossalai/zero/sharded_model 2023-02-13T02:03:10,144 copying colossalai/zero/sharded_model/__init__.py -> build/lib/colossalai/zero/sharded_model 2023-02-13T02:03:10,147 copying colossalai/zero/sharded_model/reduce_scatter.py -> build/lib/colossalai/zero/sharded_model 2023-02-13T02:03:10,151 copying colossalai/zero/sharded_model/utils.py -> build/lib/colossalai/zero/sharded_model 2023-02-13T02:03:10,155 copying colossalai/zero/sharded_model/sharded_model_v2.py -> build/lib/colossalai/zero/sharded_model 2023-02-13T02:03:10,160 copying colossalai/zero/sharded_model/_utils.py -> build/lib/colossalai/zero/sharded_model 2023-02-13T02:03:10,165 creating build/lib/colossalai/zero/shard_utils 2023-02-13T02:03:10,166 copying colossalai/zero/shard_utils/__init__.py -> build/lib/colossalai/zero/shard_utils 2023-02-13T02:03:10,170 copying colossalai/zero/shard_utils/commons.py -> build/lib/colossalai/zero/shard_utils 2023-02-13T02:03:10,173 copying colossalai/zero/shard_utils/tensor_shard_strategy.py -> build/lib/colossalai/zero/shard_utils 2023-02-13T02:03:10,176 copying colossalai/zero/shard_utils/bucket_tensor_shard_strategy.py -> build/lib/colossalai/zero/shard_utils 2023-02-13T02:03:10,179 copying colossalai/zero/shard_utils/base_shard_strategy.py -> build/lib/colossalai/zero/shard_utils 2023-02-13T02:03:10,184 creating build/lib/colossalai/zero/sharded_optim 2023-02-13T02:03:10,185 copying colossalai/zero/sharded_optim/__init__.py -> build/lib/colossalai/zero/sharded_optim 2023-02-13T02:03:10,189 copying colossalai/zero/sharded_optim/low_level_optim.py -> build/lib/colossalai/zero/sharded_optim 2023-02-13T02:03:10,194 copying colossalai/zero/sharded_optim/sharded_optim_v2.py -> build/lib/colossalai/zero/sharded_optim 2023-02-13T02:03:10,199 copying colossalai/zero/sharded_optim/_utils.py -> build/lib/colossalai/zero/sharded_optim 2023-02-13T02:03:10,204 creating build/lib/colossalai/zero/init_ctx 2023-02-13T02:03:10,205 copying colossalai/zero/init_ctx/__init__.py -> build/lib/colossalai/zero/init_ctx 2023-02-13T02:03:10,209 copying colossalai/zero/init_ctx/init_context.py -> build/lib/colossalai/zero/init_ctx 2023-02-13T02:03:10,214 creating build/lib/colossalai/zero/utils 2023-02-13T02:03:10,216 copying colossalai/zero/utils/__init__.py -> build/lib/colossalai/zero/utils 2023-02-13T02:03:10,219 copying colossalai/zero/utils/zero_hook.py -> build/lib/colossalai/zero/utils 2023-02-13T02:03:10,223 copying colossalai/zero/utils/gemini_hook.py -> build/lib/colossalai/zero/utils 2023-02-13T02:03:10,228 creating build/lib/colossalai/zero/sharded_param 2023-02-13T02:03:10,229 copying colossalai/zero/sharded_param/sharded_param.py -> build/lib/colossalai/zero/sharded_param 2023-02-13T02:03:10,233 copying colossalai/zero/sharded_param/__init__.py -> build/lib/colossalai/zero/sharded_param 2023-02-13T02:03:10,236 copying colossalai/zero/sharded_param/sharded_tensor.py -> build/lib/colossalai/zero/sharded_param 2023-02-13T02:03:10,241 creating build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:10,243 copying colossalai/zero/sharded_optim/bookkeeping/bucket_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:10,247 copying colossalai/zero/sharded_optim/bookkeeping/__init__.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:10,250 copying colossalai/zero/sharded_optim/bookkeeping/base_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:10,253 copying colossalai/zero/sharded_optim/bookkeeping/parameter_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:10,256 copying colossalai/zero/sharded_optim/bookkeeping/tensor_bucket.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:10,260 copying colossalai/zero/sharded_optim/bookkeeping/gradient_store.py -> build/lib/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:10,264 creating build/lib/colossalai/amp/apex_amp 2023-02-13T02:03:10,265 copying colossalai/amp/apex_amp/__init__.py -> build/lib/colossalai/amp/apex_amp 2023-02-13T02:03:10,269 copying colossalai/amp/apex_amp/apex_amp.py -> build/lib/colossalai/amp/apex_amp 2023-02-13T02:03:10,273 creating build/lib/colossalai/amp/naive_amp 2023-02-13T02:03:10,274 copying colossalai/amp/naive_amp/__init__.py -> build/lib/colossalai/amp/naive_amp 2023-02-13T02:03:10,277 copying colossalai/amp/naive_amp/_fp16_optimizer.py -> build/lib/colossalai/amp/naive_amp 2023-02-13T02:03:10,281 copying colossalai/amp/naive_amp/naive_amp.py -> build/lib/colossalai/amp/naive_amp 2023-02-13T02:03:10,285 copying colossalai/amp/naive_amp/_utils.py -> build/lib/colossalai/amp/naive_amp 2023-02-13T02:03:10,289 creating build/lib/colossalai/amp/torch_amp 2023-02-13T02:03:10,291 copying colossalai/amp/torch_amp/__init__.py -> build/lib/colossalai/amp/torch_amp 2023-02-13T02:03:10,294 copying colossalai/amp/torch_amp/_grad_scaler.py -> build/lib/colossalai/amp/torch_amp 2023-02-13T02:03:10,299 copying colossalai/amp/torch_amp/torch_amp.py -> build/lib/colossalai/amp/torch_amp 2023-02-13T02:03:10,303 creating build/lib/colossalai/amp/naive_amp/grad_scaler 2023-02-13T02:03:10,305 copying colossalai/amp/naive_amp/grad_scaler/__init__.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-02-13T02:03:10,308 copying colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-02-13T02:03:10,312 copying colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-02-13T02:03:10,315 copying colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py -> build/lib/colossalai/amp/naive_amp/grad_scaler 2023-02-13T02:03:10,320 creating build/lib/colossalai/utils/data_sampler 2023-02-13T02:03:10,321 copying colossalai/utils/data_sampler/__init__.py -> build/lib/colossalai/utils/data_sampler 2023-02-13T02:03:10,325 copying colossalai/utils/data_sampler/base_sampler.py -> build/lib/colossalai/utils/data_sampler 2023-02-13T02:03:10,328 copying colossalai/utils/data_sampler/data_parallel_sampler.py -> build/lib/colossalai/utils/data_sampler 2023-02-13T02:03:10,333 creating build/lib/colossalai/utils/tensor_detector 2023-02-13T02:03:10,335 copying colossalai/utils/tensor_detector/__init__.py -> build/lib/colossalai/utils/tensor_detector 2023-02-13T02:03:10,338 copying colossalai/utils/tensor_detector/tensor_detector.py -> build/lib/colossalai/utils/tensor_detector 2023-02-13T02:03:10,344 creating build/lib/colossalai/utils/rank_recorder 2023-02-13T02:03:10,345 copying colossalai/utils/rank_recorder/__init__.py -> build/lib/colossalai/utils/rank_recorder 2023-02-13T02:03:10,348 copying colossalai/utils/rank_recorder/rank_recorder.py -> build/lib/colossalai/utils/rank_recorder 2023-02-13T02:03:10,354 creating build/lib/colossalai/utils/multi_tensor_apply 2023-02-13T02:03:10,355 copying colossalai/utils/multi_tensor_apply/__init__.py -> build/lib/colossalai/utils/multi_tensor_apply 2023-02-13T02:03:10,359 copying colossalai/utils/multi_tensor_apply/multi_tensor_apply.py -> build/lib/colossalai/utils/multi_tensor_apply 2023-02-13T02:03:10,364 creating build/lib/colossalai/utils/profiler 2023-02-13T02:03:10,365 copying colossalai/utils/profiler/__init__.py -> build/lib/colossalai/utils/profiler 2023-02-13T02:03:10,369 copying colossalai/utils/profiler/extention.py -> build/lib/colossalai/utils/profiler 2023-02-13T02:03:10,372 copying colossalai/utils/profiler/profiler.py -> build/lib/colossalai/utils/profiler 2023-02-13T02:03:10,377 copying colossalai/utils/profiler/stateful_tensor_mem_extention.py -> build/lib/colossalai/utils/profiler 2023-02-13T02:03:10,382 creating build/lib/colossalai/utils/model 2023-02-13T02:03:10,383 copying colossalai/utils/model/__init__.py -> build/lib/colossalai/utils/model 2023-02-13T02:03:10,386 copying colossalai/utils/model/lazy_init_context.py -> build/lib/colossalai/utils/model 2023-02-13T02:03:10,390 copying colossalai/utils/model/colo_init_context.py -> build/lib/colossalai/utils/model 2023-02-13T02:03:10,395 copying colossalai/utils/model/utils.py -> build/lib/colossalai/utils/model 2023-02-13T02:03:10,399 copying colossalai/utils/model/experimental.py -> build/lib/colossalai/utils/model 2023-02-13T02:03:10,404 creating build/lib/colossalai/utils/checkpoint 2023-02-13T02:03:10,406 copying colossalai/utils/checkpoint/__init__.py -> build/lib/colossalai/utils/checkpoint 2023-02-13T02:03:10,409 copying colossalai/utils/checkpoint/module_checkpoint.py -> build/lib/colossalai/utils/checkpoint 2023-02-13T02:03:10,414 copying colossalai/utils/checkpoint/utils.py -> build/lib/colossalai/utils/checkpoint 2023-02-13T02:03:10,419 creating build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,421 copying colossalai/utils/checkpoint_io/convertor.py -> build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,425 copying colossalai/utils/checkpoint_io/meta.py -> build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,428 copying colossalai/utils/checkpoint_io/__init__.py -> build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,432 copying colossalai/utils/checkpoint_io/io.py -> build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,436 copying colossalai/utils/checkpoint_io/distributed.py -> build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,440 copying colossalai/utils/checkpoint_io/utils.py -> build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,445 copying colossalai/utils/checkpoint_io/backend.py -> build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,448 copying colossalai/utils/checkpoint_io/reader.py -> build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,452 copying colossalai/utils/checkpoint_io/writer.py -> build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,456 copying colossalai/utils/checkpoint_io/constant.py -> build/lib/colossalai/utils/checkpoint_io 2023-02-13T02:03:10,461 creating build/lib/colossalai/utils/profiler/legacy 2023-02-13T02:03:10,462 copying colossalai/utils/profiler/legacy/__init__.py -> build/lib/colossalai/utils/profiler/legacy 2023-02-13T02:03:10,466 copying colossalai/utils/profiler/legacy/prof_utils.py -> build/lib/colossalai/utils/profiler/legacy 2023-02-13T02:03:10,469 copying colossalai/utils/profiler/legacy/comm_profiler.py -> build/lib/colossalai/utils/profiler/legacy 2023-02-13T02:03:10,474 copying colossalai/utils/profiler/legacy/pcie_profiler.py -> build/lib/colossalai/utils/profiler/legacy 2023-02-13T02:03:10,479 creating build/lib/colossalai/kernel/cuda_native 2023-02-13T02:03:10,481 copying colossalai/kernel/cuda_native/__init__.py -> build/lib/colossalai/kernel/cuda_native 2023-02-13T02:03:10,485 copying colossalai/kernel/cuda_native/multihead_attention.py -> build/lib/colossalai/kernel/cuda_native 2023-02-13T02:03:10,489 copying colossalai/kernel/cuda_native/layer_norm.py -> build/lib/colossalai/kernel/cuda_native 2023-02-13T02:03:10,492 copying colossalai/kernel/cuda_native/flash_attention.py -> build/lib/colossalai/kernel/cuda_native 2023-02-13T02:03:10,496 copying colossalai/kernel/cuda_native/scaled_softmax.py -> build/lib/colossalai/kernel/cuda_native 2023-02-13T02:03:10,502 creating build/lib/colossalai/kernel/jit 2023-02-13T02:03:10,503 copying colossalai/kernel/jit/__init__.py -> build/lib/colossalai/kernel/jit 2023-02-13T02:03:10,506 copying colossalai/kernel/jit/bias_dropout_add.py -> build/lib/colossalai/kernel/jit 2023-02-13T02:03:10,509 copying colossalai/kernel/jit/bias_gelu.py -> build/lib/colossalai/kernel/jit 2023-02-13T02:03:10,513 copying colossalai/kernel/jit/option.py -> build/lib/colossalai/kernel/jit 2023-02-13T02:03:10,518 creating build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,519 copying colossalai/kernel/op_builder/cpu_adam.py -> build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,523 copying colossalai/kernel/op_builder/__init__.py -> build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,526 copying colossalai/kernel/op_builder/scaled_masked_softmax.py -> build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,529 copying colossalai/kernel/op_builder/moe.py -> build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,532 copying colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py -> build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,535 copying colossalai/kernel/op_builder/builder.py -> build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,539 copying colossalai/kernel/op_builder/multi_head_attn.py -> build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,542 copying colossalai/kernel/op_builder/utils.py -> build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,545 copying colossalai/kernel/op_builder/layernorm.py -> build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,549 copying colossalai/kernel/op_builder/fused_optim.py -> build/lib/colossalai/kernel/op_builder 2023-02-13T02:03:10,553 creating build/lib/colossalai/fx/codegen 2023-02-13T02:03:10,555 copying colossalai/fx/codegen/__init__.py -> build/lib/colossalai/fx/codegen 2023-02-13T02:03:10,558 copying colossalai/fx/codegen/activation_checkpoint_codegen.py -> build/lib/colossalai/fx/codegen 2023-02-13T02:03:10,566 creating build/lib/colossalai/fx/tracer 2023-02-13T02:03:10,568 copying colossalai/fx/tracer/__init__.py -> build/lib/colossalai/fx/tracer 2023-02-13T02:03:10,571 copying colossalai/fx/tracer/registry.py -> build/lib/colossalai/fx/tracer 2023-02-13T02:03:10,575 copying colossalai/fx/tracer/_meta_trace.py -> build/lib/colossalai/fx/tracer 2023-02-13T02:03:10,579 copying colossalai/fx/tracer/_tracer_utils.py -> build/lib/colossalai/fx/tracer 2023-02-13T02:03:10,582 copying colossalai/fx/tracer/experimental.py -> build/lib/colossalai/fx/tracer 2023-02-13T02:03:10,587 copying colossalai/fx/tracer/_symbolic_trace.py -> build/lib/colossalai/fx/tracer 2023-02-13T02:03:10,590 copying colossalai/fx/tracer/tracer.py -> build/lib/colossalai/fx/tracer 2023-02-13T02:03:10,597 creating build/lib/colossalai/fx/profiler 2023-02-13T02:03:10,599 copying colossalai/fx/profiler/__init__.py -> build/lib/colossalai/fx/profiler 2023-02-13T02:03:10,602 copying colossalai/fx/profiler/opcount.py -> build/lib/colossalai/fx/profiler 2023-02-13T02:03:10,607 copying colossalai/fx/profiler/constants.py -> build/lib/colossalai/fx/profiler 2023-02-13T02:03:10,610 copying colossalai/fx/profiler/dataflow.py -> build/lib/colossalai/fx/profiler 2023-02-13T02:03:10,614 copying colossalai/fx/profiler/profiler.py -> build/lib/colossalai/fx/profiler 2023-02-13T02:03:10,619 copying colossalai/fx/profiler/shard_utils.py -> build/lib/colossalai/fx/profiler 2023-02-13T02:03:10,622 copying colossalai/fx/profiler/tensor.py -> build/lib/colossalai/fx/profiler 2023-02-13T02:03:10,626 copying colossalai/fx/profiler/memory_utils.py -> build/lib/colossalai/fx/profiler 2023-02-13T02:03:10,631 creating build/lib/colossalai/fx/passes 2023-02-13T02:03:10,633 copying colossalai/fx/passes/concrete_info_prop.py -> build/lib/colossalai/fx/passes 2023-02-13T02:03:10,637 copying colossalai/fx/passes/__init__.py -> build/lib/colossalai/fx/passes 2023-02-13T02:03:10,641 copying colossalai/fx/passes/shard_1d_pass.py -> build/lib/colossalai/fx/passes 2023-02-13T02:03:10,645 copying colossalai/fx/passes/passes_for_gpt2_test.py -> build/lib/colossalai/fx/passes 2023-02-13T02:03:10,649 copying colossalai/fx/passes/meta_info_prop.py -> build/lib/colossalai/fx/passes 2023-02-13T02:03:10,653 copying colossalai/fx/passes/utils.py -> build/lib/colossalai/fx/passes 2023-02-13T02:03:10,658 copying colossalai/fx/passes/adding_split_node_pass.py -> build/lib/colossalai/fx/passes 2023-02-13T02:03:10,662 copying colossalai/fx/passes/split_module.py -> build/lib/colossalai/fx/passes 2023-02-13T02:03:10,667 creating build/lib/colossalai/fx/tracer/meta_patch 2023-02-13T02:03:10,669 copying colossalai/fx/tracer/meta_patch/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch 2023-02-13T02:03:10,674 creating build/lib/colossalai/fx/tracer/bias_addition_patch 2023-02-13T02:03:10,675 copying colossalai/fx/tracer/bias_addition_patch/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch 2023-02-13T02:03:10,681 creating build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:10,682 copying colossalai/fx/tracer/meta_patch/patched_module/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:10,686 copying colossalai/fx/tracer/meta_patch/patched_module/linear.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:10,690 copying colossalai/fx/tracer/meta_patch/patched_module/normalization.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:10,693 copying colossalai/fx/tracer/meta_patch/patched_module/convolution.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:10,697 copying colossalai/fx/tracer/meta_patch/patched_module/pooling.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:10,702 copying colossalai/fx/tracer/meta_patch/patched_module/rnn.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:10,705 copying colossalai/fx/tracer/meta_patch/patched_module/activation_function.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:10,709 copying colossalai/fx/tracer/meta_patch/patched_module/embedding.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:10,714 creating build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:10,716 copying colossalai/fx/tracer/meta_patch/patched_function/__init__.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:10,720 copying colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:10,724 copying colossalai/fx/tracer/meta_patch/patched_function/normalization.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:10,727 copying colossalai/fx/tracer/meta_patch/patched_function/convolution.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:10,731 copying colossalai/fx/tracer/meta_patch/patched_function/activation_function.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:10,735 copying colossalai/fx/tracer/meta_patch/patched_function/python_ops.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:10,739 copying colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:10,742 copying colossalai/fx/tracer/meta_patch/patched_function/embedding.py -> build/lib/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:10,747 creating build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:10,749 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:10,753 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:10,757 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:10,760 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:10,764 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:10,768 creating build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-02-13T02:03:10,770 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-02-13T02:03:10,774 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-02-13T02:03:10,777 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-02-13T02:03:10,781 copying colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py -> build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-02-13T02:03:10,786 creating build/lib/colossalai/fx/profiler/experimental 2023-02-13T02:03:10,788 copying colossalai/fx/profiler/experimental/__init__.py -> build/lib/colossalai/fx/profiler/experimental 2023-02-13T02:03:10,791 copying colossalai/fx/profiler/experimental/registry.py -> build/lib/colossalai/fx/profiler/experimental 2023-02-13T02:03:10,795 copying colossalai/fx/profiler/experimental/constants.py -> build/lib/colossalai/fx/profiler/experimental 2023-02-13T02:03:10,799 copying colossalai/fx/profiler/experimental/profiler.py -> build/lib/colossalai/fx/profiler/experimental 2023-02-13T02:03:10,803 copying colossalai/fx/profiler/experimental/shard_utils.py -> build/lib/colossalai/fx/profiler/experimental 2023-02-13T02:03:10,808 creating build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:10,809 copying colossalai/fx/profiler/experimental/profiler_function/__init__.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:10,813 copying colossalai/fx/profiler/experimental/profiler_function/linear.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:10,817 copying colossalai/fx/profiler/experimental/profiler_function/torch_ops.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:10,820 copying colossalai/fx/profiler/experimental/profiler_function/normalization.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:10,824 copying colossalai/fx/profiler/experimental/profiler_function/pooling.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:10,828 copying colossalai/fx/profiler/experimental/profiler_function/activation_function.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:10,831 copying colossalai/fx/profiler/experimental/profiler_function/python_ops.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:10,835 copying colossalai/fx/profiler/experimental/profiler_function/arithmetic.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:10,839 copying colossalai/fx/profiler/experimental/profiler_function/embedding.py -> build/lib/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:10,845 creating build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,846 copying colossalai/fx/profiler/experimental/profiler_module/__init__.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,850 copying colossalai/fx/profiler/experimental/profiler_module/linear.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,854 copying colossalai/fx/profiler/experimental/profiler_module/normalization.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,857 copying colossalai/fx/profiler/experimental/profiler_module/convolution.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,862 copying colossalai/fx/profiler/experimental/profiler_module/pooling.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,865 copying colossalai/fx/profiler/experimental/profiler_module/torch_op.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,869 copying colossalai/fx/profiler/experimental/profiler_module/rnn.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,873 copying colossalai/fx/profiler/experimental/profiler_module/dropout.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,878 copying colossalai/fx/profiler/experimental/profiler_module/activation_function.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,882 copying colossalai/fx/profiler/experimental/profiler_module/embedding.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,885 copying colossalai/fx/profiler/experimental/profiler_module/attention.py -> build/lib/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:10,890 creating build/lib/colossalai/fx/passes/algorithms 2023-02-13T02:03:10,892 copying colossalai/fx/passes/algorithms/ckpt_solver_rotor.py -> build/lib/colossalai/fx/passes/algorithms 2023-02-13T02:03:10,897 copying colossalai/fx/passes/algorithms/__init__.py -> build/lib/colossalai/fx/passes/algorithms 2023-02-13T02:03:10,900 copying colossalai/fx/passes/algorithms/linearize.py -> build/lib/colossalai/fx/passes/algorithms 2023-02-13T02:03:10,904 copying colossalai/fx/passes/algorithms/ckpt_solver_chen.py -> build/lib/colossalai/fx/passes/algorithms 2023-02-13T02:03:10,907 copying colossalai/fx/passes/algorithms/build_c_ext.py -> build/lib/colossalai/fx/passes/algorithms 2023-02-13T02:03:10,911 copying colossalai/fx/passes/algorithms/operation.py -> build/lib/colossalai/fx/passes/algorithms 2023-02-13T02:03:10,915 copying colossalai/fx/passes/algorithms/ckpt_solver_pofo.py -> build/lib/colossalai/fx/passes/algorithms 2023-02-13T02:03:10,921 creating build/lib/colossalai/pipeline/middleware 2023-02-13T02:03:10,923 copying colossalai/pipeline/middleware/topo.py -> build/lib/colossalai/pipeline/middleware 2023-02-13T02:03:10,927 copying colossalai/pipeline/middleware/__init__.py -> build/lib/colossalai/pipeline/middleware 2023-02-13T02:03:10,932 creating build/lib/colossalai/pipeline/rpc 2023-02-13T02:03:10,934 copying colossalai/pipeline/rpc/__init__.py -> build/lib/colossalai/pipeline/rpc 2023-02-13T02:03:10,937 copying colossalai/pipeline/rpc/utils.py -> build/lib/colossalai/pipeline/rpc 2023-02-13T02:03:10,942 copying colossalai/pipeline/rpc/_pipeline_base.py -> build/lib/colossalai/pipeline/rpc 2023-02-13T02:03:10,949 copying colossalai/pipeline/rpc/_pipeline_schedule.py -> build/lib/colossalai/pipeline/rpc 2023-02-13T02:03:10,955 creating build/lib/colossalai/pipeline/middleware/adaptor 2023-02-13T02:03:10,957 copying colossalai/pipeline/middleware/adaptor/__init__.py -> build/lib/colossalai/pipeline/middleware/adaptor 2023-02-13T02:03:10,961 copying colossalai/pipeline/middleware/adaptor/fx.py -> build/lib/colossalai/pipeline/middleware/adaptor 2023-02-13T02:03:10,967 creating build/lib/colossalai/cli/benchmark 2023-02-13T02:03:10,968 copying colossalai/cli/benchmark/benchmark.py -> build/lib/colossalai/cli/benchmark 2023-02-13T02:03:10,973 copying colossalai/cli/benchmark/__init__.py -> build/lib/colossalai/cli/benchmark 2023-02-13T02:03:10,976 copying colossalai/cli/benchmark/models.py -> build/lib/colossalai/cli/benchmark 2023-02-13T02:03:10,979 copying colossalai/cli/benchmark/utils.py -> build/lib/colossalai/cli/benchmark 2023-02-13T02:03:10,985 creating build/lib/colossalai/cli/launcher 2023-02-13T02:03:10,986 copying colossalai/cli/launcher/__init__.py -> build/lib/colossalai/cli/launcher 2023-02-13T02:03:10,990 copying colossalai/cli/launcher/hostinfo.py -> build/lib/colossalai/cli/launcher 2023-02-13T02:03:10,993 copying colossalai/cli/launcher/run.py -> build/lib/colossalai/cli/launcher 2023-02-13T02:03:10,997 copying colossalai/cli/launcher/multinode_runner.py -> build/lib/colossalai/cli/launcher 2023-02-13T02:03:11,002 creating build/lib/colossalai/cli/check 2023-02-13T02:03:11,004 copying colossalai/cli/check/__init__.py -> build/lib/colossalai/cli/check 2023-02-13T02:03:11,007 copying colossalai/cli/check/check_installation.py -> build/lib/colossalai/cli/check 2023-02-13T02:03:11,045 creating build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,046 copying colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,051 copying colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,057 copying colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,061 copying colossalai/kernel/cuda_native/csrc/type_shim.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,066 copying colossalai/kernel/cuda_native/csrc/cpu_adam.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,070 copying colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,075 copying colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,079 copying colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,083 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,089 copying colossalai/kernel/cuda_native/csrc/moe_cuda.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,093 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,098 copying colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,105 copying colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,110 copying colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,115 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,120 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,124 copying colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,129 copying colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,134 copying colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,140 copying colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,146 copying colossalai/kernel/cuda_native/csrc/cpu_adam.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,151 copying colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,156 copying colossalai/kernel/cuda_native/csrc/compat.h -> build/lib/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:11,160 creating build/lib/colossalai/kernel/cuda_native/csrc/kernels 2023-02-13T02:03:11,162 creating build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,164 copying colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,169 copying colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,174 copying colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,179 copying colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,184 copying colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,188 copying colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,193 copying colossalai/kernel/cuda_native/csrc/kernels/include/context.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,198 copying colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,203 copying colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,208 copying colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,213 copying colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,218 copying colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh -> build/lib/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:11,451 /usr/local/lib/python3.7/dist-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. 2023-02-13T02:03:11,451 setuptools.SetuptoolsDeprecationWarning, 2023-02-13T02:03:11,517 installing to build/bdist.linux-armv7l/wheel 2023-02-13T02:03:11,518 running install 2023-02-13T02:03:11,582 running install_lib 2023-02-13T02:03:11,653 creating build/bdist.linux-armv7l 2023-02-13T02:03:11,654 creating build/bdist.linux-armv7l/wheel 2023-02-13T02:03:11,657 creating build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,659 copying build/lib/op_builder/cpu_adam.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,662 copying build/lib/op_builder/__init__.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,666 copying build/lib/op_builder/scaled_masked_softmax.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,669 copying build/lib/op_builder/moe.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,672 copying build/lib/op_builder/scaled_upper_triangle_masked_softmax.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,676 copying build/lib/op_builder/builder.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,680 copying build/lib/op_builder/multi_head_attn.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,683 copying build/lib/op_builder/utils.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,686 copying build/lib/op_builder/layernorm.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,690 copying build/lib/op_builder/fused_optim.py -> build/bdist.linux-armv7l/wheel/op_builder 2023-02-13T02:03:11,695 creating build/bdist.linux-armv7l/wheel/colossalai 2023-02-13T02:03:11,697 creating build/bdist.linux-armv7l/wheel/colossalai/engine 2023-02-13T02:03:11,699 copying build/lib/colossalai/engine/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine 2023-02-13T02:03:11,704 creating build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-02-13T02:03:11,705 copying build/lib/colossalai/engine/schedule/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-02-13T02:03:11,709 copying build/lib/colossalai/engine/schedule/_pipeline_schedule_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-02-13T02:03:11,713 copying build/lib/colossalai/engine/schedule/_non_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-02-13T02:03:11,716 copying build/lib/colossalai/engine/schedule/_base_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-02-13T02:03:11,720 copying build/lib/colossalai/engine/schedule/_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/schedule 2023-02-13T02:03:11,727 copying build/lib/colossalai/engine/_base_engine.py -> build/bdist.linux-armv7l/wheel/colossalai/engine 2023-02-13T02:03:11,732 creating build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-02-13T02:03:11,734 copying build/lib/colossalai/engine/gradient_handler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-02-13T02:03:11,737 copying build/lib/colossalai/engine/gradient_handler/_data_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-02-13T02:03:11,740 copying build/lib/colossalai/engine/gradient_handler/_pipeline_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-02-13T02:03:11,744 copying build/lib/colossalai/engine/gradient_handler/_base_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-02-13T02:03:11,747 copying build/lib/colossalai/engine/gradient_handler/_moe_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-02-13T02:03:11,750 copying build/lib/colossalai/engine/gradient_handler/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-02-13T02:03:11,753 copying build/lib/colossalai/engine/gradient_handler/_sequence_parallel_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-02-13T02:03:11,756 copying build/lib/colossalai/engine/gradient_handler/_zero_gradient_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_handler 2023-02-13T02:03:11,761 creating build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_accumulation 2023-02-13T02:03:11,762 copying build/lib/colossalai/engine/gradient_accumulation/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_accumulation 2023-02-13T02:03:11,766 copying build/lib/colossalai/engine/gradient_accumulation/_gradient_accumulation.py -> build/bdist.linux-armv7l/wheel/colossalai/engine/gradient_accumulation 2023-02-13T02:03:11,771 creating build/bdist.linux-armv7l/wheel/colossalai/context 2023-02-13T02:03:11,772 copying build/lib/colossalai/context/moe_context.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-02-13T02:03:11,776 copying build/lib/colossalai/context/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-02-13T02:03:11,779 copying build/lib/colossalai/context/config.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-02-13T02:03:11,782 copying build/lib/colossalai/context/parallel_context.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-02-13T02:03:11,788 creating build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,790 copying build/lib/colossalai/context/process_group_initializer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,794 copying build/lib/colossalai/context/process_group_initializer/initializer_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,798 copying build/lib/colossalai/context/process_group_initializer/process_group_initializer.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,801 copying build/lib/colossalai/context/process_group_initializer/initializer_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,805 copying build/lib/colossalai/context/process_group_initializer/initializer_data.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,808 copying build/lib/colossalai/context/process_group_initializer/initializer_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,812 copying build/lib/colossalai/context/process_group_initializer/initializer_sequence.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,815 copying build/lib/colossalai/context/process_group_initializer/initializer_pipeline.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,818 copying build/lib/colossalai/context/process_group_initializer/initializer_1d.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,821 copying build/lib/colossalai/context/process_group_initializer/initializer_model.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,825 copying build/lib/colossalai/context/process_group_initializer/initializer_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/context/process_group_initializer 2023-02-13T02:03:11,828 copying build/lib/colossalai/context/parallel_mode.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-02-13T02:03:11,832 creating build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-02-13T02:03:11,834 copying build/lib/colossalai/context/random/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-02-13T02:03:11,837 copying build/lib/colossalai/context/random/seed_manager.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-02-13T02:03:11,840 copying build/lib/colossalai/context/random/_helper.py -> build/bdist.linux-armv7l/wheel/colossalai/context/random 2023-02-13T02:03:11,844 copying build/lib/colossalai/context/singleton_meta.py -> build/bdist.linux-armv7l/wheel/colossalai/context 2023-02-13T02:03:11,848 creating build/bdist.linux-armv7l/wheel/colossalai/trainer 2023-02-13T02:03:11,849 copying build/lib/colossalai/trainer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer 2023-02-13T02:03:11,853 copying build/lib/colossalai/trainer/_trainer.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer 2023-02-13T02:03:11,859 creating build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-02-13T02:03:11,860 copying build/lib/colossalai/trainer/hooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-02-13T02:03:11,864 copying build/lib/colossalai/trainer/hooks/_metric_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-02-13T02:03:11,868 copying build/lib/colossalai/trainer/hooks/_lr_scheduler_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-02-13T02:03:11,871 copying build/lib/colossalai/trainer/hooks/_checkpoint_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-02-13T02:03:11,874 copying build/lib/colossalai/trainer/hooks/_commons_.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-02-13T02:03:11,877 copying build/lib/colossalai/trainer/hooks/_base_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-02-13T02:03:11,880 copying build/lib/colossalai/trainer/hooks/_log_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/trainer/hooks 2023-02-13T02:03:11,885 copying build/lib/colossalai/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-02-13T02:03:11,888 copying build/lib/colossalai/constants.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-02-13T02:03:11,893 creating build/bdist.linux-armv7l/wheel/colossalai/builder 2023-02-13T02:03:11,894 copying build/lib/colossalai/builder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/builder 2023-02-13T02:03:11,898 copying build/lib/colossalai/builder/builder.py -> build/bdist.linux-armv7l/wheel/colossalai/builder 2023-02-13T02:03:11,902 creating build/bdist.linux-armv7l/wheel/colossalai/_C 2023-02-13T02:03:11,904 copying build/lib/colossalai/_C/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/_C 2023-02-13T02:03:11,908 creating build/bdist.linux-armv7l/wheel/colossalai/registry 2023-02-13T02:03:11,909 copying build/lib/colossalai/registry/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/registry 2023-02-13T02:03:11,913 copying build/lib/colossalai/registry/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/registry 2023-02-13T02:03:11,917 creating build/bdist.linux-armv7l/wheel/colossalai/logging 2023-02-13T02:03:11,919 copying build/lib/colossalai/logging/logger.py -> build/bdist.linux-armv7l/wheel/colossalai/logging 2023-02-13T02:03:11,923 copying build/lib/colossalai/logging/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/logging 2023-02-13T02:03:11,927 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel 2023-02-13T02:03:11,930 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-02-13T02:03:11,932 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:11,934 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/solver.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:11,939 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:11,942 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:11,945 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/graph_analysis.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:11,949 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/sharding_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:11,953 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/cost_graph.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:11,957 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/strategies_constructor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:11,963 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/options.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:11,966 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated 2023-02-13T02:03:11,971 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:11,973 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:11,976 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/operator_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:11,980 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/batch_norm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:11,985 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/unary_elementwise_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:11,989 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/reshape_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:11,993 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/embedding_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:11,997 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/where_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:12,002 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/layer_norm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:12,006 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:12,010 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/bcast_op_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:12,015 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/dot_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:12,020 copying build/lib/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/conv_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/deprecated/op_handler 2023-02-13T02:03:12,026 copying build/lib/colossalai/auto_parallel/tensor_shard/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-02-13T02:03:12,029 copying build/lib/colossalai/auto_parallel/tensor_shard/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-02-13T02:03:12,033 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:12,035 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/solver.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:12,040 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:12,043 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:12,047 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/cost_graph.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:12,051 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:12,055 copying build/lib/colossalai/auto_parallel/tensor_shard/solver/options.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/solver 2023-02-13T02:03:12,060 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,061 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,065 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,068 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,072 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,076 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,080 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,084 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,087 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,090 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,094 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,097 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,100 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,104 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,107 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,114 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,115 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,120 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,124 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,128 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,131 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,136 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,140 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,143 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,147 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,151 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,155 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,159 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,164 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,169 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,173 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,177 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,181 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,184 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,189 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler/strategy 2023-02-13T02:03:12,193 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,196 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,200 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,204 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,207 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,211 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,215 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,218 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/option.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,221 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,224 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,228 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,231 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,235 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,239 copying build/lib/colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/node_handler 2023-02-13T02:03:12,244 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:12,246 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/misc.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:12,249 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:12,253 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/broadcast.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:12,257 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/reshape.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:12,261 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/factory.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:12,265 copying build/lib/colossalai/auto_parallel/tensor_shard/utils/sharding.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard/utils 2023-02-13T02:03:12,268 copying build/lib/colossalai/auto_parallel/tensor_shard/sharding_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-02-13T02:03:12,272 copying build/lib/colossalai/auto_parallel/tensor_shard/initialize.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/tensor_shard 2023-02-13T02:03:12,277 copying build/lib/colossalai/auto_parallel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel 2023-02-13T02:03:12,280 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/pipeline_shard 2023-02-13T02:03:12,282 copying build/lib/colossalai/auto_parallel/pipeline_shard/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/pipeline_shard 2023-02-13T02:03:12,286 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-02-13T02:03:12,287 copying build/lib/colossalai/auto_parallel/meta_profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-02-13T02:03:12,290 copying build/lib/colossalai/auto_parallel/meta_profiler/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-02-13T02:03:12,294 copying build/lib/colossalai/auto_parallel/meta_profiler/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-02-13T02:03:12,297 copying build/lib/colossalai/auto_parallel/meta_profiler/metainfo.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler 2023-02-13T02:03:12,301 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:12,303 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:12,306 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:12,310 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:12,314 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/activation.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:12,318 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/conv.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:12,322 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/norm.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:12,326 copying build/lib/colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/meta_profiler/meta_registry 2023-02-13T02:03:12,331 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-02-13T02:03:12,332 copying build/lib/colossalai/auto_parallel/passes/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-02-13T02:03:12,335 copying build/lib/colossalai/auto_parallel/passes/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-02-13T02:03:12,339 copying build/lib/colossalai/auto_parallel/passes/runtime_apply_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-02-13T02:03:12,342 copying build/lib/colossalai/auto_parallel/passes/meta_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-02-13T02:03:12,347 copying build/lib/colossalai/auto_parallel/passes/runtime_preparation_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-02-13T02:03:12,351 copying build/lib/colossalai/auto_parallel/passes/comm_metainfo_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/passes 2023-02-13T02:03:12,356 creating build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:12,358 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:12,363 copying build/lib/colossalai/auto_parallel/checkpoint/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:12,366 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:12,369 copying build/lib/colossalai/auto_parallel/checkpoint/build_c_ext.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:12,373 copying build/lib/colossalai/auto_parallel/checkpoint/operation.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:12,377 copying build/lib/colossalai/auto_parallel/checkpoint/ckpt_solver_base.py -> build/bdist.linux-armv7l/wheel/colossalai/auto_parallel/checkpoint 2023-02-13T02:03:12,382 creating build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-02-13T02:03:12,383 copying build/lib/colossalai/gemini/stateful_tensor_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-02-13T02:03:12,387 copying build/lib/colossalai/gemini/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-02-13T02:03:12,391 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/paramhooks 2023-02-13T02:03:12,393 copying build/lib/colossalai/gemini/paramhooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/paramhooks 2023-02-13T02:03:12,397 copying build/lib/colossalai/gemini/paramhooks/_param_hookmgr.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/paramhooks 2023-02-13T02:03:12,400 copying build/lib/colossalai/gemini/gemini_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-02-13T02:03:12,404 copying build/lib/colossalai/gemini/gemini_context.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-02-13T02:03:12,407 copying build/lib/colossalai/gemini/placement_policy.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-02-13T02:03:12,412 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-02-13T02:03:12,414 copying build/lib/colossalai/gemini/memory_tracer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-02-13T02:03:12,417 copying build/lib/colossalai/gemini/memory_tracer/static_memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-02-13T02:03:12,421 copying build/lib/colossalai/gemini/memory_tracer/runtime_mem_tracer.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-02-13T02:03:12,425 copying build/lib/colossalai/gemini/memory_tracer/memory_monitor.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-02-13T02:03:12,428 copying build/lib/colossalai/gemini/memory_tracer/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-02-13T02:03:12,431 copying build/lib/colossalai/gemini/memory_tracer/memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-02-13T02:03:12,435 copying build/lib/colossalai/gemini/memory_tracer/param_runtime_order.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-02-13T02:03:12,438 copying build/lib/colossalai/gemini/memory_tracer/memory_stats.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-02-13T02:03:12,442 copying build/lib/colossalai/gemini/memory_tracer/chunk_memstats_collector.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/memory_tracer 2023-02-13T02:03:12,446 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-02-13T02:03:12,447 copying build/lib/colossalai/gemini/chunk/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-02-13T02:03:12,451 copying build/lib/colossalai/gemini/chunk/chunk.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-02-13T02:03:12,456 copying build/lib/colossalai/gemini/chunk/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-02-13T02:03:12,459 copying build/lib/colossalai/gemini/chunk/manager.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-02-13T02:03:12,464 copying build/lib/colossalai/gemini/chunk/search_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/chunk 2023-02-13T02:03:12,468 copying build/lib/colossalai/gemini/stateful_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-02-13T02:03:12,471 copying build/lib/colossalai/gemini/tensor_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-02-13T02:03:12,476 creating build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-02-13T02:03:12,478 copying build/lib/colossalai/gemini/ophooks/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-02-13T02:03:12,481 copying build/lib/colossalai/gemini/ophooks/_shard_param_ophook.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-02-13T02:03:12,485 copying build/lib/colossalai/gemini/ophooks/_shard_grad_ophook.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-02-13T02:03:12,488 copying build/lib/colossalai/gemini/ophooks/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-02-13T02:03:12,492 copying build/lib/colossalai/gemini/ophooks/runtime_mem_tracer_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini/ophooks 2023-02-13T02:03:12,496 copying build/lib/colossalai/gemini/tensor_placement_policy.py -> build/bdist.linux-armv7l/wheel/colossalai/gemini 2023-02-13T02:03:12,502 creating build/bdist.linux-armv7l/wheel/colossalai/nn 2023-02-13T02:03:12,504 copying build/lib/colossalai/nn/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn 2023-02-13T02:03:12,508 creating build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,510 copying build/lib/colossalai/nn/_ops/loss.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,514 copying build/lib/colossalai/nn/_ops/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,517 copying build/lib/colossalai/nn/_ops/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,521 copying build/lib/colossalai/nn/_ops/embedding_bag.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,525 copying build/lib/colossalai/nn/_ops/view.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,528 copying build/lib/colossalai/nn/_ops/batch_norm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,531 copying build/lib/colossalai/nn/_ops/addmm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,535 copying build/lib/colossalai/nn/_ops/element_wise.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,539 copying build/lib/colossalai/nn/_ops/layernorm.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,542 copying build/lib/colossalai/nn/_ops/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,546 copying build/lib/colossalai/nn/_ops/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/_ops 2023-02-13T02:03:12,552 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-02-13T02:03:12,554 copying build/lib/colossalai/nn/parallel/gemini_parallel.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-02-13T02:03:12,558 copying build/lib/colossalai/nn/parallel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-02-13T02:03:12,561 copying build/lib/colossalai/nn/parallel/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-02-13T02:03:12,566 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-02-13T02:03:12,567 copying build/lib/colossalai/nn/parallel/layers/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-02-13T02:03:12,571 copying build/lib/colossalai/nn/parallel/layers/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-02-13T02:03:12,576 creating build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:12,577 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:12,581 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:12,585 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:12,589 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/embedding_config.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:12,593 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/copyer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:12,596 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:12,600 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/base_embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:12,604 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:12,608 copying build/lib/colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers/cache_embedding 2023-02-13T02:03:12,614 copying build/lib/colossalai/nn/parallel/layers/module_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-02-13T02:03:12,618 copying build/lib/colossalai/nn/parallel/layers/colo_module.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-02-13T02:03:12,621 copying build/lib/colossalai/nn/parallel/layers/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel/layers 2023-02-13T02:03:12,625 copying build/lib/colossalai/nn/parallel/data_parallel.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-02-13T02:03:12,630 copying build/lib/colossalai/nn/parallel/reducer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-02-13T02:03:12,634 copying build/lib/colossalai/nn/parallel/zero_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/parallel 2023-02-13T02:03:12,639 creating build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-02-13T02:03:12,641 copying build/lib/colossalai/nn/loss/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-02-13T02:03:12,646 copying build/lib/colossalai/nn/loss/loss_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-02-13T02:03:12,650 copying build/lib/colossalai/nn/loss/loss_moe.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-02-13T02:03:12,654 copying build/lib/colossalai/nn/loss/loss_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-02-13T02:03:12,659 copying build/lib/colossalai/nn/loss/loss_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-02-13T02:03:12,663 copying build/lib/colossalai/nn/loss/loss_1d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/loss 2023-02-13T02:03:12,668 copying build/lib/colossalai/nn/init.py -> build/bdist.linux-armv7l/wheel/colossalai/nn 2023-02-13T02:03:12,674 creating build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-02-13T02:03:12,676 copying build/lib/colossalai/nn/lr_scheduler/cosine.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-02-13T02:03:12,680 copying build/lib/colossalai/nn/lr_scheduler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-02-13T02:03:12,685 copying build/lib/colossalai/nn/lr_scheduler/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-02-13T02:03:12,689 copying build/lib/colossalai/nn/lr_scheduler/onecycle.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-02-13T02:03:12,693 copying build/lib/colossalai/nn/lr_scheduler/poly.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-02-13T02:03:12,697 copying build/lib/colossalai/nn/lr_scheduler/torch.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-02-13T02:03:12,701 copying build/lib/colossalai/nn/lr_scheduler/delayed.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-02-13T02:03:12,706 copying build/lib/colossalai/nn/lr_scheduler/multistep.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/lr_scheduler 2023-02-13T02:03:12,711 creating build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,714 copying build/lib/colossalai/nn/optimizer/hybrid_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,718 copying build/lib/colossalai/nn/optimizer/fused_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,723 copying build/lib/colossalai/nn/optimizer/cpu_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,728 copying build/lib/colossalai/nn/optimizer/lars.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,732 copying build/lib/colossalai/nn/optimizer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,736 copying build/lib/colossalai/nn/optimizer/fused_lamb.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,741 copying build/lib/colossalai/nn/optimizer/gemini_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,745 copying build/lib/colossalai/nn/optimizer/fused_sgd.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,749 copying build/lib/colossalai/nn/optimizer/colossalai_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,753 copying build/lib/colossalai/nn/optimizer/zero_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,758 copying build/lib/colossalai/nn/optimizer/nvme_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,763 copying build/lib/colossalai/nn/optimizer/lamb.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/optimizer 2023-02-13T02:03:12,769 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-02-13T02:03:12,771 copying build/lib/colossalai/nn/layer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-02-13T02:03:12,776 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-02-13T02:03:12,778 copying build/lib/colossalai/nn/layer/parallel_2d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-02-13T02:03:12,785 copying build/lib/colossalai/nn/layer/parallel_2d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-02-13T02:03:12,789 copying build/lib/colossalai/nn/layer/parallel_2d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-02-13T02:03:12,794 copying build/lib/colossalai/nn/layer/parallel_2d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2d 2023-02-13T02:03:12,799 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:12,800 copying build/lib/colossalai/nn/layer/colossalai_layer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:12,804 copying build/lib/colossalai/nn/layer/colossalai_layer/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:12,808 copying build/lib/colossalai/nn/layer/colossalai_layer/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:12,811 copying build/lib/colossalai/nn/layer/colossalai_layer/dropout.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:12,814 copying build/lib/colossalai/nn/layer/colossalai_layer/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:12,818 copying build/lib/colossalai/nn/layer/colossalai_layer/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/colossalai_layer 2023-02-13T02:03:12,823 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-02-13T02:03:12,824 copying build/lib/colossalai/nn/layer/parallel_2p5d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-02-13T02:03:12,831 copying build/lib/colossalai/nn/layer/parallel_2p5d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-02-13T02:03:12,834 copying build/lib/colossalai/nn/layer/parallel_2p5d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-02-13T02:03:12,840 copying build/lib/colossalai/nn/layer/parallel_2p5d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_2p5d 2023-02-13T02:03:12,845 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-02-13T02:03:12,846 copying build/lib/colossalai/nn/layer/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-02-13T02:03:12,850 copying build/lib/colossalai/nn/layer/utils/common.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/utils 2023-02-13T02:03:12,854 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-02-13T02:03:12,856 copying build/lib/colossalai/nn/layer/parallel_3d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-02-13T02:03:12,863 copying build/lib/colossalai/nn/layer/parallel_3d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-02-13T02:03:12,866 copying build/lib/colossalai/nn/layer/parallel_3d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-02-13T02:03:12,871 copying build/lib/colossalai/nn/layer/parallel_3d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_3d 2023-02-13T02:03:12,876 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-02-13T02:03:12,878 copying build/lib/colossalai/nn/layer/moe/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-02-13T02:03:12,883 copying build/lib/colossalai/nn/layer/moe/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-02-13T02:03:12,887 copying build/lib/colossalai/nn/layer/moe/experts.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-02-13T02:03:12,892 copying build/lib/colossalai/nn/layer/moe/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-02-13T02:03:12,896 copying build/lib/colossalai/nn/layer/moe/routers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-02-13T02:03:12,901 copying build/lib/colossalai/nn/layer/moe/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/moe 2023-02-13T02:03:12,907 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-02-13T02:03:12,909 copying build/lib/colossalai/nn/layer/parallel_sequence/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-02-13T02:03:12,915 copying build/lib/colossalai/nn/layer/parallel_sequence/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-02-13T02:03:12,919 copying build/lib/colossalai/nn/layer/parallel_sequence/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-02-13T02:03:12,923 copying build/lib/colossalai/nn/layer/parallel_sequence/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_sequence 2023-02-13T02:03:12,928 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-02-13T02:03:12,931 copying build/lib/colossalai/nn/layer/parallel_1d/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-02-13T02:03:12,938 copying build/lib/colossalai/nn/layer/parallel_1d/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-02-13T02:03:12,942 copying build/lib/colossalai/nn/layer/parallel_1d/_operation.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-02-13T02:03:12,947 copying build/lib/colossalai/nn/layer/parallel_1d/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/parallel_1d 2023-02-13T02:03:12,952 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-02-13T02:03:12,954 copying build/lib/colossalai/nn/layer/wrapper/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-02-13T02:03:12,959 copying build/lib/colossalai/nn/layer/wrapper/pipeline_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/wrapper 2023-02-13T02:03:12,964 creating build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-02-13T02:03:12,966 copying build/lib/colossalai/nn/layer/vanilla/layers.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-02-13T02:03:12,972 copying build/lib/colossalai/nn/layer/vanilla/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer/vanilla 2023-02-13T02:03:12,976 copying build/lib/colossalai/nn/layer/base_layer.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/layer 2023-02-13T02:03:12,981 creating build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-02-13T02:03:12,983 copying build/lib/colossalai/nn/metric/accuracy_2p5d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-02-13T02:03:12,987 copying build/lib/colossalai/nn/metric/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-02-13T02:03:12,991 copying build/lib/colossalai/nn/metric/accuracy_2d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-02-13T02:03:12,995 copying build/lib/colossalai/nn/metric/accuracy_3d.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-02-13T02:03:12,999 copying build/lib/colossalai/nn/metric/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/nn/metric 2023-02-13T02:03:13,003 copying build/lib/colossalai/core.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-02-13T02:03:13,009 creating build/bdist.linux-armv7l/wheel/colossalai/zero 2023-02-13T02:03:13,011 copying build/lib/colossalai/zero/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero 2023-02-13T02:03:13,016 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-02-13T02:03:13,018 copying build/lib/colossalai/zero/sharded_model/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-02-13T02:03:13,022 copying build/lib/colossalai/zero/sharded_model/reduce_scatter.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-02-13T02:03:13,027 copying build/lib/colossalai/zero/sharded_model/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-02-13T02:03:13,030 copying build/lib/colossalai/zero/sharded_model/sharded_model_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-02-13T02:03:13,036 copying build/lib/colossalai/zero/sharded_model/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_model 2023-02-13T02:03:13,040 creating build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-02-13T02:03:13,041 copying build/lib/colossalai/zero/shard_utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-02-13T02:03:13,045 copying build/lib/colossalai/zero/shard_utils/commons.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-02-13T02:03:13,048 copying build/lib/colossalai/zero/shard_utils/tensor_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-02-13T02:03:13,052 copying build/lib/colossalai/zero/shard_utils/bucket_tensor_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-02-13T02:03:13,055 copying build/lib/colossalai/zero/shard_utils/base_shard_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/shard_utils 2023-02-13T02:03:13,060 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-02-13T02:03:13,061 copying build/lib/colossalai/zero/sharded_optim/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-02-13T02:03:13,066 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:13,068 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/bucket_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:13,071 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:13,075 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/base_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:13,078 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/parameter_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:13,081 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/tensor_bucket.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:13,085 copying build/lib/colossalai/zero/sharded_optim/bookkeeping/gradient_store.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim/bookkeeping 2023-02-13T02:03:13,088 copying build/lib/colossalai/zero/sharded_optim/low_level_optim.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-02-13T02:03:13,093 copying build/lib/colossalai/zero/sharded_optim/sharded_optim_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-02-13T02:03:13,097 copying build/lib/colossalai/zero/sharded_optim/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_optim 2023-02-13T02:03:13,103 creating build/bdist.linux-armv7l/wheel/colossalai/zero/init_ctx 2023-02-13T02:03:13,104 copying build/lib/colossalai/zero/init_ctx/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/init_ctx 2023-02-13T02:03:13,107 copying build/lib/colossalai/zero/init_ctx/init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/init_ctx 2023-02-13T02:03:13,113 creating build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-02-13T02:03:13,115 copying build/lib/colossalai/zero/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-02-13T02:03:13,118 copying build/lib/colossalai/zero/utils/zero_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-02-13T02:03:13,122 copying build/lib/colossalai/zero/utils/gemini_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/utils 2023-02-13T02:03:13,127 creating build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-02-13T02:03:13,128 copying build/lib/colossalai/zero/sharded_param/sharded_param.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-02-13T02:03:13,132 copying build/lib/colossalai/zero/sharded_param/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-02-13T02:03:13,136 copying build/lib/colossalai/zero/sharded_param/sharded_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/zero/sharded_param 2023-02-13T02:03:13,141 creating build/bdist.linux-armv7l/wheel/colossalai/amp 2023-02-13T02:03:13,143 copying build/lib/colossalai/amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp 2023-02-13T02:03:13,149 creating build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-02-13T02:03:13,151 copying build/lib/colossalai/amp/apex_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-02-13T02:03:13,155 copying build/lib/colossalai/amp/apex_amp/apex_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/apex_amp 2023-02-13T02:03:13,160 creating build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-02-13T02:03:13,164 creating build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-02-13T02:03:13,166 copying build/lib/colossalai/amp/naive_amp/grad_scaler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-02-13T02:03:13,170 copying build/lib/colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-02-13T02:03:13,175 copying build/lib/colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-02-13T02:03:13,179 copying build/lib/colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp/grad_scaler 2023-02-13T02:03:13,183 copying build/lib/colossalai/amp/naive_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-02-13T02:03:13,187 copying build/lib/colossalai/amp/naive_amp/_fp16_optimizer.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-02-13T02:03:13,192 copying build/lib/colossalai/amp/naive_amp/naive_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-02-13T02:03:13,197 copying build/lib/colossalai/amp/naive_amp/_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/naive_amp 2023-02-13T02:03:13,201 copying build/lib/colossalai/amp/amp_type.py -> build/bdist.linux-armv7l/wheel/colossalai/amp 2023-02-13T02:03:13,206 creating build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-02-13T02:03:13,208 copying build/lib/colossalai/amp/torch_amp/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-02-13T02:03:13,212 copying build/lib/colossalai/amp/torch_amp/_grad_scaler.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-02-13T02:03:13,218 copying build/lib/colossalai/amp/torch_amp/torch_amp.py -> build/bdist.linux-armv7l/wheel/colossalai/amp/torch_amp 2023-02-13T02:03:13,223 creating build/bdist.linux-armv7l/wheel/colossalai/utils 2023-02-13T02:03:13,226 creating build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-02-13T02:03:13,227 copying build/lib/colossalai/utils/data_sampler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-02-13T02:03:13,231 copying build/lib/colossalai/utils/data_sampler/base_sampler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-02-13T02:03:13,235 copying build/lib/colossalai/utils/data_sampler/data_parallel_sampler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/data_sampler 2023-02-13T02:03:13,240 creating build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-02-13T02:03:13,242 copying build/lib/colossalai/utils/tensor_detector/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-02-13T02:03:13,245 copying build/lib/colossalai/utils/tensor_detector/tensor_detector.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/tensor_detector 2023-02-13T02:03:13,249 copying build/lib/colossalai/utils/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-02-13T02:03:13,253 copying build/lib/colossalai/utils/moe.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-02-13T02:03:13,257 creating build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-02-13T02:03:13,259 copying build/lib/colossalai/utils/rank_recorder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-02-13T02:03:13,262 copying build/lib/colossalai/utils/rank_recorder/rank_recorder.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/rank_recorder 2023-02-13T02:03:13,266 copying build/lib/colossalai/utils/timer.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-02-13T02:03:13,271 creating build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-02-13T02:03:13,273 copying build/lib/colossalai/utils/multi_tensor_apply/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-02-13T02:03:13,276 copying build/lib/colossalai/utils/multi_tensor_apply/multi_tensor_apply.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/multi_tensor_apply 2023-02-13T02:03:13,280 copying build/lib/colossalai/utils/cuda.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-02-13T02:03:13,285 creating build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-02-13T02:03:13,286 copying build/lib/colossalai/utils/profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-02-13T02:03:13,290 copying build/lib/colossalai/utils/profiler/extention.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-02-13T02:03:13,293 copying build/lib/colossalai/utils/profiler/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-02-13T02:03:13,297 copying build/lib/colossalai/utils/profiler/stateful_tensor_mem_extention.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler 2023-02-13T02:03:13,302 creating build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-02-13T02:03:13,303 copying build/lib/colossalai/utils/profiler/legacy/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-02-13T02:03:13,307 copying build/lib/colossalai/utils/profiler/legacy/prof_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-02-13T02:03:13,311 copying build/lib/colossalai/utils/profiler/legacy/comm_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-02-13T02:03:13,315 copying build/lib/colossalai/utils/profiler/legacy/pcie_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/profiler/legacy 2023-02-13T02:03:13,319 copying build/lib/colossalai/utils/common.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-02-13T02:03:13,324 copying build/lib/colossalai/utils/checkpointing.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-02-13T02:03:13,330 creating build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-02-13T02:03:13,331 copying build/lib/colossalai/utils/model/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-02-13T02:03:13,334 copying build/lib/colossalai/utils/model/lazy_init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-02-13T02:03:13,339 copying build/lib/colossalai/utils/model/colo_init_context.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-02-13T02:03:13,344 copying build/lib/colossalai/utils/model/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-02-13T02:03:13,347 copying build/lib/colossalai/utils/model/experimental.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/model 2023-02-13T02:03:13,352 copying build/lib/colossalai/utils/memory.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-02-13T02:03:13,357 creating build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-02-13T02:03:13,358 copying build/lib/colossalai/utils/checkpoint/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-02-13T02:03:13,362 copying build/lib/colossalai/utils/checkpoint/module_checkpoint.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-02-13T02:03:13,367 copying build/lib/colossalai/utils/checkpoint/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint 2023-02-13T02:03:13,371 creating build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,373 copying build/lib/colossalai/utils/checkpoint_io/convertor.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,377 copying build/lib/colossalai/utils/checkpoint_io/meta.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,381 copying build/lib/colossalai/utils/checkpoint_io/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,384 copying build/lib/colossalai/utils/checkpoint_io/io.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,388 copying build/lib/colossalai/utils/checkpoint_io/distributed.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,392 copying build/lib/colossalai/utils/checkpoint_io/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,396 copying build/lib/colossalai/utils/checkpoint_io/backend.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,400 copying build/lib/colossalai/utils/checkpoint_io/reader.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,404 copying build/lib/colossalai/utils/checkpoint_io/writer.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,408 copying build/lib/colossalai/utils/checkpoint_io/constant.py -> build/bdist.linux-armv7l/wheel/colossalai/utils/checkpoint_io 2023-02-13T02:03:13,411 copying build/lib/colossalai/utils/activation_checkpoint.py -> build/bdist.linux-armv7l/wheel/colossalai/utils 2023-02-13T02:03:13,416 creating build/bdist.linux-armv7l/wheel/colossalai/communication 2023-02-13T02:03:13,418 copying build/lib/colossalai/communication/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-02-13T02:03:13,421 copying build/lib/colossalai/communication/p2p_v2.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-02-13T02:03:13,425 copying build/lib/colossalai/communication/ring.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-02-13T02:03:13,429 copying build/lib/colossalai/communication/p2p.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-02-13T02:03:13,434 copying build/lib/colossalai/communication/collective.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-02-13T02:03:13,438 copying build/lib/colossalai/communication/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/communication 2023-02-13T02:03:13,443 creating build/bdist.linux-armv7l/wheel/colossalai/kernel 2023-02-13T02:03:13,445 copying build/lib/colossalai/kernel/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel 2023-02-13T02:03:13,449 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-02-13T02:03:13,451 copying build/lib/colossalai/kernel/cuda_native/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-02-13T02:03:13,456 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,457 copying build/lib/colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,461 copying build/lib/colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,467 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,471 copying build/lib/colossalai/kernel/cuda_native/csrc/type_shim.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,476 copying build/lib/colossalai/kernel/cuda_native/csrc/cpu_adam.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,480 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,484 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,489 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,494 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels 2023-02-13T02:03:13,496 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,498 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,502 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,506 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,510 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,513 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,517 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,520 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/context.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,523 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,527 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,531 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,534 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,538 copying build/lib/colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc/kernels/include 2023-02-13T02:03:13,541 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,546 copying build/lib/colossalai/kernel/cuda_native/csrc/moe_cuda.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,550 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,555 copying build/lib/colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,560 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,564 copying build/lib/colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,568 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,572 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,575 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,579 copying build/lib/colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,583 copying build/lib/colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,587 copying build/lib/colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,592 copying build/lib/colossalai/kernel/cuda_native/csrc/cpu_adam.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,596 copying build/lib/colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,600 copying build/lib/colossalai/kernel/cuda_native/csrc/compat.h -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native/csrc 2023-02-13T02:03:13,603 copying build/lib/colossalai/kernel/cuda_native/multihead_attention.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-02-13T02:03:13,607 copying build/lib/colossalai/kernel/cuda_native/layer_norm.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-02-13T02:03:13,611 copying build/lib/colossalai/kernel/cuda_native/flash_attention.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-02-13T02:03:13,615 copying build/lib/colossalai/kernel/cuda_native/scaled_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/cuda_native 2023-02-13T02:03:13,620 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-02-13T02:03:13,622 copying build/lib/colossalai/kernel/jit/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-02-13T02:03:13,625 copying build/lib/colossalai/kernel/jit/bias_dropout_add.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-02-13T02:03:13,629 copying build/lib/colossalai/kernel/jit/bias_gelu.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-02-13T02:03:13,632 copying build/lib/colossalai/kernel/jit/option.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/jit 2023-02-13T02:03:13,636 creating build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,638 copying build/lib/colossalai/kernel/op_builder/cpu_adam.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,641 copying build/lib/colossalai/kernel/op_builder/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,645 copying build/lib/colossalai/kernel/op_builder/scaled_masked_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,648 copying build/lib/colossalai/kernel/op_builder/moe.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,651 copying build/lib/colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,655 copying build/lib/colossalai/kernel/op_builder/builder.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,660 copying build/lib/colossalai/kernel/op_builder/multi_head_attn.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,663 copying build/lib/colossalai/kernel/op_builder/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,667 copying build/lib/colossalai/kernel/op_builder/layernorm.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,670 copying build/lib/colossalai/kernel/op_builder/fused_optim.py -> build/bdist.linux-armv7l/wheel/colossalai/kernel/op_builder 2023-02-13T02:03:13,674 creating build/bdist.linux-armv7l/wheel/colossalai/device 2023-02-13T02:03:13,676 copying build/lib/colossalai/device/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-02-13T02:03:13,679 copying build/lib/colossalai/device/alpha_beta_profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-02-13T02:03:13,685 copying build/lib/colossalai/device/device_mesh.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-02-13T02:03:13,689 copying build/lib/colossalai/device/calc_pipeline_strategy.py -> build/bdist.linux-armv7l/wheel/colossalai/device 2023-02-13T02:03:13,694 creating build/bdist.linux-armv7l/wheel/colossalai/fx 2023-02-13T02:03:13,696 copying build/lib/colossalai/fx/graph_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-02-13T02:03:13,700 copying build/lib/colossalai/fx/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-02-13T02:03:13,705 creating build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-02-13T02:03:13,707 copying build/lib/colossalai/fx/codegen/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-02-13T02:03:13,710 copying build/lib/colossalai/fx/codegen/activation_checkpoint_codegen.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/codegen 2023-02-13T02:03:13,716 copying build/lib/colossalai/fx/_meta_registrations.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-02-13T02:03:13,722 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-02-13T02:03:13,724 copying build/lib/colossalai/fx/tracer/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-02-13T02:03:13,728 copying build/lib/colossalai/fx/tracer/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-02-13T02:03:13,731 copying build/lib/colossalai/fx/tracer/_meta_trace.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-02-13T02:03:13,735 copying build/lib/colossalai/fx/tracer/_tracer_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-02-13T02:03:13,740 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch 2023-02-13T02:03:13,741 copying build/lib/colossalai/fx/tracer/meta_patch/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch 2023-02-13T02:03:13,746 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:13,748 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:13,752 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:13,755 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:13,759 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:13,763 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:13,767 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/rnn.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:13,771 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:13,774 copying build/lib/colossalai/fx/tracer/meta_patch/patched_module/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_module 2023-02-13T02:03:13,779 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:13,781 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:13,784 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:13,788 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:13,792 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:13,796 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:13,799 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/python_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:13,803 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:13,806 copying build/lib/colossalai/fx/tracer/meta_patch/patched_function/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/meta_patch/patched_function 2023-02-13T02:03:13,811 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch 2023-02-13T02:03:13,812 copying build/lib/colossalai/fx/tracer/bias_addition_patch/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch 2023-02-13T02:03:13,817 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:13,819 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:13,823 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:13,826 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:13,829 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:13,833 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function 2023-02-13T02:03:13,837 creating build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-02-13T02:03:13,839 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-02-13T02:03:13,842 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-02-13T02:03:13,846 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-02-13T02:03:13,849 copying build/lib/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module 2023-02-13T02:03:13,853 copying build/lib/colossalai/fx/tracer/experimental.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-02-13T02:03:13,858 copying build/lib/colossalai/fx/tracer/_symbolic_trace.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-02-13T02:03:13,861 copying build/lib/colossalai/fx/tracer/tracer.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/tracer 2023-02-13T02:03:13,866 copying build/lib/colossalai/fx/_compatibility.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-02-13T02:03:13,870 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-02-13T02:03:13,873 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-02-13T02:03:13,874 copying build/lib/colossalai/fx/profiler/experimental/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-02-13T02:03:13,878 copying build/lib/colossalai/fx/profiler/experimental/registry.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-02-13T02:03:13,881 copying build/lib/colossalai/fx/profiler/experimental/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-02-13T02:03:13,885 copying build/lib/colossalai/fx/profiler/experimental/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-02-13T02:03:13,888 copying build/lib/colossalai/fx/profiler/experimental/shard_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental 2023-02-13T02:03:13,893 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:13,894 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:13,897 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:13,900 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/torch_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:13,904 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:13,907 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:13,910 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:13,913 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/python_ops.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:13,916 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/arithmetic.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:13,920 copying build/lib/colossalai/fx/profiler/experimental/profiler_function/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_function 2023-02-13T02:03:13,924 creating build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,926 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,929 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/linear.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,932 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/normalization.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,936 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/convolution.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,939 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/pooling.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,942 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/torch_op.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,945 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/rnn.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,949 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/dropout.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,952 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/activation_function.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,955 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/embedding.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,958 copying build/lib/colossalai/fx/profiler/experimental/profiler_module/attention.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler/experimental/profiler_module 2023-02-13T02:03:13,961 copying build/lib/colossalai/fx/profiler/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-02-13T02:03:13,964 copying build/lib/colossalai/fx/profiler/opcount.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-02-13T02:03:13,968 copying build/lib/colossalai/fx/profiler/constants.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-02-13T02:03:13,971 copying build/lib/colossalai/fx/profiler/dataflow.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-02-13T02:03:13,975 copying build/lib/colossalai/fx/profiler/profiler.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-02-13T02:03:13,980 copying build/lib/colossalai/fx/profiler/shard_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-02-13T02:03:13,984 copying build/lib/colossalai/fx/profiler/tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-02-13T02:03:13,988 copying build/lib/colossalai/fx/profiler/memory_utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/profiler 2023-02-13T02:03:13,992 creating build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-02-13T02:03:13,994 copying build/lib/colossalai/fx/passes/concrete_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-02-13T02:03:13,999 creating build/bdist.linux-armv7l/wheel/colossalai/fx/passes/algorithms 2023-02-13T02:03:14,000 copying build/lib/colossalai/fx/passes/algorithms/ckpt_solver_rotor.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes/algorithms 2023-02-13T02:03:14,005 copying build/lib/colossalai/fx/passes/algorithms/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes/algorithms 2023-02-13T02:03:14,008 copying build/lib/colossalai/fx/passes/algorithms/linearize.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes/algorithms 2023-02-13T02:03:14,012 copying build/lib/colossalai/fx/passes/algorithms/ckpt_solver_chen.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes/algorithms 2023-02-13T02:03:14,015 copying build/lib/colossalai/fx/passes/algorithms/build_c_ext.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes/algorithms 2023-02-13T02:03:14,018 copying build/lib/colossalai/fx/passes/algorithms/operation.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes/algorithms 2023-02-13T02:03:14,022 copying build/lib/colossalai/fx/passes/algorithms/ckpt_solver_pofo.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes/algorithms 2023-02-13T02:03:14,028 copying build/lib/colossalai/fx/passes/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-02-13T02:03:14,031 copying build/lib/colossalai/fx/passes/shard_1d_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-02-13T02:03:14,035 copying build/lib/colossalai/fx/passes/passes_for_gpt2_test.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-02-13T02:03:14,039 copying build/lib/colossalai/fx/passes/meta_info_prop.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-02-13T02:03:14,044 copying build/lib/colossalai/fx/passes/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-02-13T02:03:14,047 copying build/lib/colossalai/fx/passes/adding_split_node_pass.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-02-13T02:03:14,051 copying build/lib/colossalai/fx/passes/split_module.py -> build/bdist.linux-armv7l/wheel/colossalai/fx/passes 2023-02-13T02:03:14,058 copying build/lib/colossalai/fx/proxy.py -> build/bdist.linux-armv7l/wheel/colossalai/fx 2023-02-13T02:03:14,063 creating build/bdist.linux-armv7l/wheel/colossalai/testing 2023-02-13T02:03:14,065 copying build/lib/colossalai/testing/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-02-13T02:03:14,069 copying build/lib/colossalai/testing/comparison.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-02-13T02:03:14,073 copying build/lib/colossalai/testing/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-02-13T02:03:14,078 copying build/lib/colossalai/testing/random.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-02-13T02:03:14,082 copying build/lib/colossalai/testing/pytest_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/testing 2023-02-13T02:03:14,087 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-02-13T02:03:14,089 copying build/lib/colossalai/pipeline/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-02-13T02:03:14,094 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-02-13T02:03:14,096 copying build/lib/colossalai/pipeline/middleware/topo.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-02-13T02:03:14,101 copying build/lib/colossalai/pipeline/middleware/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware 2023-02-13T02:03:14,106 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-02-13T02:03:14,109 copying build/lib/colossalai/pipeline/middleware/adaptor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-02-13T02:03:14,113 copying build/lib/colossalai/pipeline/middleware/adaptor/fx.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/middleware/adaptor 2023-02-13T02:03:14,117 copying build/lib/colossalai/pipeline/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-02-13T02:03:14,123 creating build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-02-13T02:03:14,126 copying build/lib/colossalai/pipeline/rpc/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-02-13T02:03:14,130 copying build/lib/colossalai/pipeline/rpc/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-02-13T02:03:14,135 copying build/lib/colossalai/pipeline/rpc/_pipeline_base.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-02-13T02:03:14,141 copying build/lib/colossalai/pipeline/rpc/_pipeline_schedule.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline/rpc 2023-02-13T02:03:14,147 copying build/lib/colossalai/pipeline/pipelinable.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-02-13T02:03:14,152 copying build/lib/colossalai/pipeline/layer_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-02-13T02:03:14,156 copying build/lib/colossalai/pipeline/pipeline_process_group.py -> build/bdist.linux-armv7l/wheel/colossalai/pipeline 2023-02-13T02:03:14,161 creating build/bdist.linux-armv7l/wheel/colossalai/cli 2023-02-13T02:03:14,164 copying build/lib/colossalai/cli/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli 2023-02-13T02:03:14,168 copying build/lib/colossalai/cli/cli.py -> build/bdist.linux-armv7l/wheel/colossalai/cli 2023-02-13T02:03:14,173 creating build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-02-13T02:03:14,175 copying build/lib/colossalai/cli/benchmark/benchmark.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-02-13T02:03:14,180 copying build/lib/colossalai/cli/benchmark/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-02-13T02:03:14,184 copying build/lib/colossalai/cli/benchmark/models.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-02-13T02:03:14,188 copying build/lib/colossalai/cli/benchmark/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/benchmark 2023-02-13T02:03:14,193 creating build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-02-13T02:03:14,195 copying build/lib/colossalai/cli/launcher/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-02-13T02:03:14,200 copying build/lib/colossalai/cli/launcher/hostinfo.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-02-13T02:03:14,204 copying build/lib/colossalai/cli/launcher/run.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-02-13T02:03:14,209 copying build/lib/colossalai/cli/launcher/multinode_runner.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/launcher 2023-02-13T02:03:14,214 creating build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-02-13T02:03:14,217 copying build/lib/colossalai/cli/check/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-02-13T02:03:14,221 copying build/lib/colossalai/cli/check/check_installation.py -> build/bdist.linux-armv7l/wheel/colossalai/cli/check 2023-02-13T02:03:14,225 copying build/lib/colossalai/initialize.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-02-13T02:03:14,231 copying build/lib/colossalai/version.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-02-13T02:03:14,235 copying build/lib/colossalai/global_variables.py -> build/bdist.linux-armv7l/wheel/colossalai 2023-02-13T02:03:14,239 creating build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,241 copying build/lib/colossalai/tensor/const.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,244 copying build/lib/colossalai/tensor/dist_spec_mgr.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,248 copying build/lib/colossalai/tensor/__init__.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,251 copying build/lib/colossalai/tensor/sharding_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,256 copying build/lib/colossalai/tensor/colo_tensor.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,260 copying build/lib/colossalai/tensor/distspec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,263 copying build/lib/colossalai/tensor/param_op_hook.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,267 copying build/lib/colossalai/tensor/utils.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,272 copying build/lib/colossalai/tensor/compute_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,275 copying build/lib/colossalai/tensor/tensor_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,278 copying build/lib/colossalai/tensor/comm_spec.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,282 copying build/lib/colossalai/tensor/colo_parameter.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,287 copying build/lib/colossalai/tensor/shape_consistency.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,292 copying build/lib/colossalai/tensor/op_wrapper.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,295 copying build/lib/colossalai/tensor/process_group.py -> build/bdist.linux-armv7l/wheel/colossalai/tensor 2023-02-13T02:03:14,299 running install_egg_info 2023-02-13T02:03:14,503 running egg_info 2023-02-13T02:03:14,567 writing colossalai.egg-info/PKG-INFO 2023-02-13T02:03:14,572 writing dependency_links to colossalai.egg-info/dependency_links.txt 2023-02-13T02:03:14,576 writing entry points to colossalai.egg-info/entry_points.txt 2023-02-13T02:03:14,578 writing requirements to colossalai.egg-info/requires.txt 2023-02-13T02:03:14,581 writing top-level names to colossalai.egg-info/top_level.txt 2023-02-13T02:03:14,847 reading manifest file 'colossalai.egg-info/SOURCES.txt' 2023-02-13T02:03:14,921 reading manifest template 'MANIFEST.in' 2023-02-13T02:03:16,995 warning: no files found matching '*.tr' under directory 'colossalai' 2023-02-13T02:03:18,021 warning: no files found matching '*.cc' under directory 'colossalai' 2023-02-13T02:03:18,525 warning: no files found matching '*.pyi' under directory 'colossalai' 2023-02-13T02:03:18,541 adding license file 'LICENSE' 2023-02-13T02:03:18,639 writing manifest file 'colossalai.egg-info/SOURCES.txt' 2023-02-13T02:03:18,645 Copying colossalai.egg-info to build/bdist.linux-armv7l/wheel/colossalai-0.2.3-py3.7.egg-info 2023-02-13T02:03:18,670 running install_scripts 2023-02-13T02:03:18,849 adding license file "LICENSE" (matched pattern "LICEN[CS]E*") 2023-02-13T02:03:18,859 creating build/bdist.linux-armv7l/wheel/colossalai-0.2.3.dist-info/WHEEL 2023-02-13T02:03:18,864 creating '/tmp/pip-wheel-g7izrfet/colossalai-0.2.3-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2023-02-13T02:03:18,870 adding 'colossalai/__init__.py' 2023-02-13T02:03:18,874 adding 'colossalai/constants.py' 2023-02-13T02:03:18,877 adding 'colossalai/core.py' 2023-02-13T02:03:18,879 adding 'colossalai/global_variables.py' 2023-02-13T02:03:18,886 adding 'colossalai/initialize.py' 2023-02-13T02:03:18,889 adding 'colossalai/version.py' 2023-02-13T02:03:18,892 adding 'colossalai/_C/__init__.py' 2023-02-13T02:03:18,897 adding 'colossalai/amp/__init__.py' 2023-02-13T02:03:18,900 adding 'colossalai/amp/amp_type.py' 2023-02-13T02:03:18,904 adding 'colossalai/amp/apex_amp/__init__.py' 2023-02-13T02:03:18,907 adding 'colossalai/amp/apex_amp/apex_amp.py' 2023-02-13T02:03:18,911 adding 'colossalai/amp/naive_amp/__init__.py' 2023-02-13T02:03:18,916 adding 'colossalai/amp/naive_amp/_fp16_optimizer.py' 2023-02-13T02:03:18,920 adding 'colossalai/amp/naive_amp/_utils.py' 2023-02-13T02:03:18,923 adding 'colossalai/amp/naive_amp/naive_amp.py' 2023-02-13T02:03:18,928 adding 'colossalai/amp/naive_amp/grad_scaler/__init__.py' 2023-02-13T02:03:18,931 adding 'colossalai/amp/naive_amp/grad_scaler/base_grad_scaler.py' 2023-02-13T02:03:18,934 adding 'colossalai/amp/naive_amp/grad_scaler/constant_grad_scaler.py' 2023-02-13T02:03:18,937 adding 'colossalai/amp/naive_amp/grad_scaler/dynamic_grad_scaler.py' 2023-02-13T02:03:18,941 adding 'colossalai/amp/torch_amp/__init__.py' 2023-02-13T02:03:18,950 adding 'colossalai/amp/torch_amp/_grad_scaler.py' 2023-02-13T02:03:18,954 adding 'colossalai/amp/torch_amp/torch_amp.py' 2023-02-13T02:03:18,958 adding 'colossalai/auto_parallel/__init__.py' 2023-02-13T02:03:18,962 adding 'colossalai/auto_parallel/checkpoint/__init__.py' 2023-02-13T02:03:18,965 adding 'colossalai/auto_parallel/checkpoint/build_c_ext.py' 2023-02-13T02:03:18,969 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_base.py' 2023-02-13T02:03:18,972 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_chen.py' 2023-02-13T02:03:18,978 adding 'colossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py' 2023-02-13T02:03:18,982 adding 'colossalai/auto_parallel/checkpoint/operation.py' 2023-02-13T02:03:18,985 adding 'colossalai/auto_parallel/meta_profiler/__init__.py' 2023-02-13T02:03:18,987 adding 'colossalai/auto_parallel/meta_profiler/constants.py' 2023-02-13T02:03:18,990 adding 'colossalai/auto_parallel/meta_profiler/metainfo.py' 2023-02-13T02:03:18,992 adding 'colossalai/auto_parallel/meta_profiler/registry.py' 2023-02-13T02:03:18,995 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/__init__.py' 2023-02-13T02:03:18,998 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/activation.py' 2023-02-13T02:03:19,001 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/binary_elementwise_ops.py' 2023-02-13T02:03:19,004 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/conv.py' 2023-02-13T02:03:19,009 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/linear.py' 2023-02-13T02:03:19,013 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/norm.py' 2023-02-13T02:03:19,016 adding 'colossalai/auto_parallel/meta_profiler/meta_registry/pooling.py' 2023-02-13T02:03:19,019 adding 'colossalai/auto_parallel/passes/__init__.py' 2023-02-13T02:03:19,022 adding 'colossalai/auto_parallel/passes/comm_metainfo_pass.py' 2023-02-13T02:03:19,024 adding 'colossalai/auto_parallel/passes/constants.py' 2023-02-13T02:03:19,027 adding 'colossalai/auto_parallel/passes/meta_info_prop.py' 2023-02-13T02:03:19,031 adding 'colossalai/auto_parallel/passes/runtime_apply_pass.py' 2023-02-13T02:03:19,037 adding 'colossalai/auto_parallel/passes/runtime_preparation_pass.py' 2023-02-13T02:03:19,040 adding 'colossalai/auto_parallel/pipeline_shard/__init__.py' 2023-02-13T02:03:19,044 adding 'colossalai/auto_parallel/tensor_shard/__init__.py' 2023-02-13T02:03:19,046 adding 'colossalai/auto_parallel/tensor_shard/constants.py' 2023-02-13T02:03:19,050 adding 'colossalai/auto_parallel/tensor_shard/initialize.py' 2023-02-13T02:03:19,054 adding 'colossalai/auto_parallel/tensor_shard/sharding_strategy.py' 2023-02-13T02:03:19,058 adding 'colossalai/auto_parallel/tensor_shard/deprecated/__init__.py' 2023-02-13T02:03:19,061 adding 'colossalai/auto_parallel/tensor_shard/deprecated/_utils.py' 2023-02-13T02:03:19,064 adding 'colossalai/auto_parallel/tensor_shard/deprecated/constants.py' 2023-02-13T02:03:19,067 adding 'colossalai/auto_parallel/tensor_shard/deprecated/cost_graph.py' 2023-02-13T02:03:19,070 adding 'colossalai/auto_parallel/tensor_shard/deprecated/graph_analysis.py' 2023-02-13T02:03:19,073 adding 'colossalai/auto_parallel/tensor_shard/deprecated/options.py' 2023-02-13T02:03:19,075 adding 'colossalai/auto_parallel/tensor_shard/deprecated/sharding_strategy.py' 2023-02-13T02:03:19,081 adding 'colossalai/auto_parallel/tensor_shard/deprecated/solver.py' 2023-02-13T02:03:19,086 adding 'colossalai/auto_parallel/tensor_shard/deprecated/strategies_constructor.py' 2023-02-13T02:03:19,090 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/__init__.py' 2023-02-13T02:03:19,095 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/batch_norm_handler.py' 2023-02-13T02:03:19,102 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/bcast_op_handler.py' 2023-02-13T02:03:19,108 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/conv_handler.py' 2023-02-13T02:03:19,115 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/dot_handler.py' 2023-02-13T02:03:19,119 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/embedding_handler.py' 2023-02-13T02:03:19,122 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/layer_norm_handler.py' 2023-02-13T02:03:19,126 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/operator_handler.py' 2023-02-13T02:03:19,129 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/reshape_handler.py' 2023-02-13T02:03:19,131 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/strategy_generator.py' 2023-02-13T02:03:19,134 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/unary_elementwise_handler.py' 2023-02-13T02:03:19,138 adding 'colossalai/auto_parallel/tensor_shard/deprecated/op_handler/where_handler.py' 2023-02-13T02:03:19,142 adding 'colossalai/auto_parallel/tensor_shard/node_handler/__init__.py' 2023-02-13T02:03:19,145 adding 'colossalai/auto_parallel/tensor_shard/node_handler/addmm_handler.py' 2023-02-13T02:03:19,148 adding 'colossalai/auto_parallel/tensor_shard/node_handler/batch_norm_handler.py' 2023-02-13T02:03:19,151 adding 'colossalai/auto_parallel/tensor_shard/node_handler/binary_elementwise_handler.py' 2023-02-13T02:03:19,154 adding 'colossalai/auto_parallel/tensor_shard/node_handler/bmm_handler.py' 2023-02-13T02:03:19,157 adding 'colossalai/auto_parallel/tensor_shard/node_handler/conv_handler.py' 2023-02-13T02:03:19,159 adding 'colossalai/auto_parallel/tensor_shard/node_handler/default_reshape_handler.py' 2023-02-13T02:03:19,163 adding 'colossalai/auto_parallel/tensor_shard/node_handler/embedding_handler.py' 2023-02-13T02:03:19,165 adding 'colossalai/auto_parallel/tensor_shard/node_handler/getattr_handler.py' 2023-02-13T02:03:19,168 adding 'colossalai/auto_parallel/tensor_shard/node_handler/getitem_handler.py' 2023-02-13T02:03:19,170 adding 'colossalai/auto_parallel/tensor_shard/node_handler/layer_norm_handler.py' 2023-02-13T02:03:19,174 adding 'colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py' 2023-02-13T02:03:19,180 adding 'colossalai/auto_parallel/tensor_shard/node_handler/matmul_handler.py' 2023-02-13T02:03:19,185 adding 'colossalai/auto_parallel/tensor_shard/node_handler/node_handler.py' 2023-02-13T02:03:19,187 adding 'colossalai/auto_parallel/tensor_shard/node_handler/normal_pooling_handler.py' 2023-02-13T02:03:19,190 adding 'colossalai/auto_parallel/tensor_shard/node_handler/option.py' 2023-02-13T02:03:19,192 adding 'colossalai/auto_parallel/tensor_shard/node_handler/output_handler.py' 2023-02-13T02:03:19,195 adding 'colossalai/auto_parallel/tensor_shard/node_handler/permute_handler.py' 2023-02-13T02:03:19,197 adding 'colossalai/auto_parallel/tensor_shard/node_handler/placeholder_handler.py' 2023-02-13T02:03:19,199 adding 'colossalai/auto_parallel/tensor_shard/node_handler/registry.py' 2023-02-13T02:03:19,202 adding 'colossalai/auto_parallel/tensor_shard/node_handler/softmax_handler.py' 2023-02-13T02:03:19,204 adding 'colossalai/auto_parallel/tensor_shard/node_handler/split_handler.py' 2023-02-13T02:03:19,207 adding 'colossalai/auto_parallel/tensor_shard/node_handler/sum_handler.py' 2023-02-13T02:03:19,209 adding 'colossalai/auto_parallel/tensor_shard/node_handler/tensor_constructor_handler.py' 2023-02-13T02:03:19,212 adding 'colossalai/auto_parallel/tensor_shard/node_handler/transpose_handler.py' 2023-02-13T02:03:19,214 adding 'colossalai/auto_parallel/tensor_shard/node_handler/unary_elementwise_handler.py' 2023-02-13T02:03:19,217 adding 'colossalai/auto_parallel/tensor_shard/node_handler/view_handler.py' 2023-02-13T02:03:19,219 adding 'colossalai/auto_parallel/tensor_shard/node_handler/where_handler.py' 2023-02-13T02:03:19,223 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/__init__.py' 2023-02-13T02:03:19,227 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/batch_norm_generator.py' 2023-02-13T02:03:19,230 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/binary_elementwise_generator.py' 2023-02-13T02:03:19,235 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/conv_strategy_generator.py' 2023-02-13T02:03:19,239 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/embedding_generator.py' 2023-02-13T02:03:19,242 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/getattr_generator.py' 2023-02-13T02:03:19,245 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/getitem_generator.py' 2023-02-13T02:03:19,248 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/layer_norm_generator.py' 2023-02-13T02:03:19,255 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py' 2023-02-13T02:03:19,259 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/normal_pooling_generator.py' 2023-02-13T02:03:19,262 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/output_generator.py' 2023-02-13T02:03:19,264 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/placeholder_generator.py' 2023-02-13T02:03:19,269 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/reshape_generator.py' 2023-02-13T02:03:19,272 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/softmax_generator.py' 2023-02-13T02:03:19,276 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/strategy_generator.py' 2023-02-13T02:03:19,279 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/sum_generator.py' 2023-02-13T02:03:19,282 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/tensor_constructor_generator.py' 2023-02-13T02:03:19,285 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/unary_elementwise_generator.py' 2023-02-13T02:03:19,287 adding 'colossalai/auto_parallel/tensor_shard/node_handler/strategy/where_generator.py' 2023-02-13T02:03:19,291 adding 'colossalai/auto_parallel/tensor_shard/solver/__init__.py' 2023-02-13T02:03:19,294 adding 'colossalai/auto_parallel/tensor_shard/solver/cost_graph.py' 2023-02-13T02:03:19,298 adding 'colossalai/auto_parallel/tensor_shard/solver/graph_analysis.py' 2023-02-13T02:03:19,300 adding 'colossalai/auto_parallel/tensor_shard/solver/options.py' 2023-02-13T02:03:19,305 adding 'colossalai/auto_parallel/tensor_shard/solver/solver.py' 2023-02-13T02:03:19,309 adding 'colossalai/auto_parallel/tensor_shard/solver/strategies_constructor.py' 2023-02-13T02:03:19,312 adding 'colossalai/auto_parallel/tensor_shard/utils/__init__.py' 2023-02-13T02:03:19,315 adding 'colossalai/auto_parallel/tensor_shard/utils/broadcast.py' 2023-02-13T02:03:19,318 adding 'colossalai/auto_parallel/tensor_shard/utils/factory.py' 2023-02-13T02:03:19,321 adding 'colossalai/auto_parallel/tensor_shard/utils/misc.py' 2023-02-13T02:03:19,325 adding 'colossalai/auto_parallel/tensor_shard/utils/reshape.py' 2023-02-13T02:03:19,328 adding 'colossalai/auto_parallel/tensor_shard/utils/sharding.py' 2023-02-13T02:03:19,331 adding 'colossalai/builder/__init__.py' 2023-02-13T02:03:19,334 adding 'colossalai/builder/builder.py' 2023-02-13T02:03:19,337 adding 'colossalai/cli/__init__.py' 2023-02-13T02:03:19,339 adding 'colossalai/cli/cli.py' 2023-02-13T02:03:19,342 adding 'colossalai/cli/benchmark/__init__.py' 2023-02-13T02:03:19,345 adding 'colossalai/cli/benchmark/benchmark.py' 2023-02-13T02:03:19,347 adding 'colossalai/cli/benchmark/models.py' 2023-02-13T02:03:19,350 adding 'colossalai/cli/benchmark/utils.py' 2023-02-13T02:03:19,353 adding 'colossalai/cli/check/__init__.py' 2023-02-13T02:03:19,357 adding 'colossalai/cli/check/check_installation.py' 2023-02-13T02:03:19,361 adding 'colossalai/cli/launcher/__init__.py' 2023-02-13T02:03:19,364 adding 'colossalai/cli/launcher/hostinfo.py' 2023-02-13T02:03:19,367 adding 'colossalai/cli/launcher/multinode_runner.py' 2023-02-13T02:03:19,371 adding 'colossalai/cli/launcher/run.py' 2023-02-13T02:03:19,374 adding 'colossalai/communication/__init__.py' 2023-02-13T02:03:19,378 adding 'colossalai/communication/collective.py' 2023-02-13T02:03:19,382 adding 'colossalai/communication/p2p.py' 2023-02-13T02:03:19,386 adding 'colossalai/communication/p2p_v2.py' 2023-02-13T02:03:19,389 adding 'colossalai/communication/ring.py' 2023-02-13T02:03:19,391 adding 'colossalai/communication/utils.py' 2023-02-13T02:03:19,395 adding 'colossalai/context/__init__.py' 2023-02-13T02:03:19,398 adding 'colossalai/context/config.py' 2023-02-13T02:03:19,400 adding 'colossalai/context/moe_context.py' 2023-02-13T02:03:19,407 adding 'colossalai/context/parallel_context.py' 2023-02-13T02:03:19,409 adding 'colossalai/context/parallel_mode.py' 2023-02-13T02:03:19,412 adding 'colossalai/context/singleton_meta.py' 2023-02-13T02:03:19,415 adding 'colossalai/context/process_group_initializer/__init__.py' 2023-02-13T02:03:19,418 adding 'colossalai/context/process_group_initializer/initializer_1d.py' 2023-02-13T02:03:19,420 adding 'colossalai/context/process_group_initializer/initializer_2d.py' 2023-02-13T02:03:19,424 adding 'colossalai/context/process_group_initializer/initializer_2p5d.py' 2023-02-13T02:03:19,427 adding 'colossalai/context/process_group_initializer/initializer_3d.py' 2023-02-13T02:03:19,430 adding 'colossalai/context/process_group_initializer/initializer_data.py' 2023-02-13T02:03:19,433 adding 'colossalai/context/process_group_initializer/initializer_model.py' 2023-02-13T02:03:19,435 adding 'colossalai/context/process_group_initializer/initializer_pipeline.py' 2023-02-13T02:03:19,438 adding 'colossalai/context/process_group_initializer/initializer_sequence.py' 2023-02-13T02:03:19,440 adding 'colossalai/context/process_group_initializer/initializer_tensor.py' 2023-02-13T02:03:19,443 adding 'colossalai/context/process_group_initializer/process_group_initializer.py' 2023-02-13T02:03:19,446 adding 'colossalai/context/random/__init__.py' 2023-02-13T02:03:19,449 adding 'colossalai/context/random/_helper.py' 2023-02-13T02:03:19,452 adding 'colossalai/context/random/seed_manager.py' 2023-02-13T02:03:19,455 adding 'colossalai/device/__init__.py' 2023-02-13T02:03:19,460 adding 'colossalai/device/alpha_beta_profiler.py' 2023-02-13T02:03:19,464 adding 'colossalai/device/calc_pipeline_strategy.py' 2023-02-13T02:03:19,467 adding 'colossalai/device/device_mesh.py' 2023-02-13T02:03:19,471 adding 'colossalai/engine/__init__.py' 2023-02-13T02:03:19,474 adding 'colossalai/engine/_base_engine.py' 2023-02-13T02:03:19,478 adding 'colossalai/engine/gradient_accumulation/__init__.py' 2023-02-13T02:03:19,481 adding 'colossalai/engine/gradient_accumulation/_gradient_accumulation.py' 2023-02-13T02:03:19,485 adding 'colossalai/engine/gradient_handler/__init__.py' 2023-02-13T02:03:19,487 adding 'colossalai/engine/gradient_handler/_base_gradient_handler.py' 2023-02-13T02:03:19,490 adding 'colossalai/engine/gradient_handler/_data_parallel_gradient_handler.py' 2023-02-13T02:03:19,492 adding 'colossalai/engine/gradient_handler/_moe_gradient_handler.py' 2023-02-13T02:03:19,495 adding 'colossalai/engine/gradient_handler/_pipeline_parallel_gradient_handler.py' 2023-02-13T02:03:19,497 adding 'colossalai/engine/gradient_handler/_sequence_parallel_gradient_handler.py' 2023-02-13T02:03:19,499 adding 'colossalai/engine/gradient_handler/_zero_gradient_handler.py' 2023-02-13T02:03:19,501 adding 'colossalai/engine/gradient_handler/utils.py' 2023-02-13T02:03:19,505 adding 'colossalai/engine/schedule/__init__.py' 2023-02-13T02:03:19,508 adding 'colossalai/engine/schedule/_base_schedule.py' 2023-02-13T02:03:19,510 adding 'colossalai/engine/schedule/_non_pipeline_schedule.py' 2023-02-13T02:03:19,519 adding 'colossalai/engine/schedule/_pipeline_schedule.py' 2023-02-13T02:03:19,523 adding 'colossalai/engine/schedule/_pipeline_schedule_v2.py' 2023-02-13T02:03:19,527 adding 'colossalai/fx/__init__.py' 2023-02-13T02:03:19,529 adding 'colossalai/fx/_compatibility.py' 2023-02-13T02:03:19,535 adding 'colossalai/fx/_meta_registrations.py' 2023-02-13T02:03:19,538 adding 'colossalai/fx/graph_module.py' 2023-02-13T02:03:19,541 adding 'colossalai/fx/proxy.py' 2023-02-13T02:03:19,544 adding 'colossalai/fx/codegen/__init__.py' 2023-02-13T02:03:19,555 adding 'colossalai/fx/codegen/activation_checkpoint_codegen.py' 2023-02-13T02:03:19,559 adding 'colossalai/fx/passes/__init__.py' 2023-02-13T02:03:19,562 adding 'colossalai/fx/passes/adding_split_node_pass.py' 2023-02-13T02:03:19,566 adding 'colossalai/fx/passes/concrete_info_prop.py' 2023-02-13T02:03:19,570 adding 'colossalai/fx/passes/meta_info_prop.py' 2023-02-13T02:03:19,576 adding 'colossalai/fx/passes/passes_for_gpt2_test.py' 2023-02-13T02:03:19,579 adding 'colossalai/fx/passes/shard_1d_pass.py' 2023-02-13T02:03:19,584 adding 'colossalai/fx/passes/split_module.py' 2023-02-13T02:03:19,587 adding 'colossalai/fx/passes/utils.py' 2023-02-13T02:03:19,590 adding 'colossalai/fx/passes/algorithms/__init__.py' 2023-02-13T02:03:19,593 adding 'colossalai/fx/passes/algorithms/build_c_ext.py' 2023-02-13T02:03:19,595 adding 'colossalai/fx/passes/algorithms/ckpt_solver_chen.py' 2023-02-13T02:03:19,601 adding 'colossalai/fx/passes/algorithms/ckpt_solver_pofo.py' 2023-02-13T02:03:19,606 adding 'colossalai/fx/passes/algorithms/ckpt_solver_rotor.py' 2023-02-13T02:03:19,609 adding 'colossalai/fx/passes/algorithms/linearize.py' 2023-02-13T02:03:19,612 adding 'colossalai/fx/passes/algorithms/operation.py' 2023-02-13T02:03:19,616 adding 'colossalai/fx/profiler/__init__.py' 2023-02-13T02:03:19,618 adding 'colossalai/fx/profiler/constants.py' 2023-02-13T02:03:19,621 adding 'colossalai/fx/profiler/dataflow.py' 2023-02-13T02:03:19,624 adding 'colossalai/fx/profiler/memory_utils.py' 2023-02-13T02:03:19,628 adding 'colossalai/fx/profiler/opcount.py' 2023-02-13T02:03:19,633 adding 'colossalai/fx/profiler/profiler.py' 2023-02-13T02:03:19,636 adding 'colossalai/fx/profiler/shard_utils.py' 2023-02-13T02:03:19,639 adding 'colossalai/fx/profiler/tensor.py' 2023-02-13T02:03:19,642 adding 'colossalai/fx/profiler/experimental/__init__.py' 2023-02-13T02:03:19,644 adding 'colossalai/fx/profiler/experimental/constants.py' 2023-02-13T02:03:19,647 adding 'colossalai/fx/profiler/experimental/profiler.py' 2023-02-13T02:03:19,650 adding 'colossalai/fx/profiler/experimental/registry.py' 2023-02-13T02:03:19,652 adding 'colossalai/fx/profiler/experimental/shard_utils.py' 2023-02-13T02:03:19,656 adding 'colossalai/fx/profiler/experimental/profiler_function/__init__.py' 2023-02-13T02:03:19,658 adding 'colossalai/fx/profiler/experimental/profiler_function/activation_function.py' 2023-02-13T02:03:19,660 adding 'colossalai/fx/profiler/experimental/profiler_function/arithmetic.py' 2023-02-13T02:03:19,663 adding 'colossalai/fx/profiler/experimental/profiler_function/embedding.py' 2023-02-13T02:03:19,665 adding 'colossalai/fx/profiler/experimental/profiler_function/linear.py' 2023-02-13T02:03:19,667 adding 'colossalai/fx/profiler/experimental/profiler_function/normalization.py' 2023-02-13T02:03:19,669 adding 'colossalai/fx/profiler/experimental/profiler_function/pooling.py' 2023-02-13T02:03:19,671 adding 'colossalai/fx/profiler/experimental/profiler_function/python_ops.py' 2023-02-13T02:03:19,674 adding 'colossalai/fx/profiler/experimental/profiler_function/torch_ops.py' 2023-02-13T02:03:19,677 adding 'colossalai/fx/profiler/experimental/profiler_module/__init__.py' 2023-02-13T02:03:19,679 adding 'colossalai/fx/profiler/experimental/profiler_module/activation_function.py' 2023-02-13T02:03:19,682 adding 'colossalai/fx/profiler/experimental/profiler_module/attention.py' 2023-02-13T02:03:19,685 adding 'colossalai/fx/profiler/experimental/profiler_module/convolution.py' 2023-02-13T02:03:19,687 adding 'colossalai/fx/profiler/experimental/profiler_module/dropout.py' 2023-02-13T02:03:19,689 adding 'colossalai/fx/profiler/experimental/profiler_module/embedding.py' 2023-02-13T02:03:19,691 adding 'colossalai/fx/profiler/experimental/profiler_module/linear.py' 2023-02-13T02:03:19,694 adding 'colossalai/fx/profiler/experimental/profiler_module/normalization.py' 2023-02-13T02:03:19,696 adding 'colossalai/fx/profiler/experimental/profiler_module/pooling.py' 2023-02-13T02:03:19,698 adding 'colossalai/fx/profiler/experimental/profiler_module/rnn.py' 2023-02-13T02:03:19,700 adding 'colossalai/fx/profiler/experimental/profiler_module/torch_op.py' 2023-02-13T02:03:19,704 adding 'colossalai/fx/tracer/__init__.py' 2023-02-13T02:03:19,707 adding 'colossalai/fx/tracer/_meta_trace.py' 2023-02-13T02:03:19,709 adding 'colossalai/fx/tracer/_symbolic_trace.py' 2023-02-13T02:03:19,712 adding 'colossalai/fx/tracer/_tracer_utils.py' 2023-02-13T02:03:19,719 adding 'colossalai/fx/tracer/experimental.py' 2023-02-13T02:03:19,722 adding 'colossalai/fx/tracer/registry.py' 2023-02-13T02:03:19,728 adding 'colossalai/fx/tracer/tracer.py' 2023-02-13T02:03:19,732 adding 'colossalai/fx/tracer/bias_addition_patch/__init__.py' 2023-02-13T02:03:19,736 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/__init__.py' 2023-02-13T02:03:19,739 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addbmm.py' 2023-02-13T02:03:19,741 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/addmm.py' 2023-02-13T02:03:19,744 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/bias_addition_function.py' 2023-02-13T02:03:19,746 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_function/linear.py' 2023-02-13T02:03:19,750 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/__init__.py' 2023-02-13T02:03:19,752 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/bias_addition_module.py' 2023-02-13T02:03:19,755 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/conv.py' 2023-02-13T02:03:19,757 adding 'colossalai/fx/tracer/bias_addition_patch/patched_bias_addition_module/linear.py' 2023-02-13T02:03:19,761 adding 'colossalai/fx/tracer/meta_patch/__init__.py' 2023-02-13T02:03:19,764 adding 'colossalai/fx/tracer/meta_patch/patched_function/__init__.py' 2023-02-13T02:03:19,766 adding 'colossalai/fx/tracer/meta_patch/patched_function/activation_function.py' 2023-02-13T02:03:19,768 adding 'colossalai/fx/tracer/meta_patch/patched_function/arithmetic.py' 2023-02-13T02:03:19,771 adding 'colossalai/fx/tracer/meta_patch/patched_function/convolution.py' 2023-02-13T02:03:19,774 adding 'colossalai/fx/tracer/meta_patch/patched_function/embedding.py' 2023-02-13T02:03:19,776 adding 'colossalai/fx/tracer/meta_patch/patched_function/normalization.py' 2023-02-13T02:03:19,778 adding 'colossalai/fx/tracer/meta_patch/patched_function/python_ops.py' 2023-02-13T02:03:19,781 adding 'colossalai/fx/tracer/meta_patch/patched_function/torch_ops.py' 2023-02-13T02:03:19,785 adding 'colossalai/fx/tracer/meta_patch/patched_module/__init__.py' 2023-02-13T02:03:19,787 adding 'colossalai/fx/tracer/meta_patch/patched_module/activation_function.py' 2023-02-13T02:03:19,789 adding 'colossalai/fx/tracer/meta_patch/patched_module/convolution.py' 2023-02-13T02:03:19,791 adding 'colossalai/fx/tracer/meta_patch/patched_module/embedding.py' 2023-02-13T02:03:19,794 adding 'colossalai/fx/tracer/meta_patch/patched_module/linear.py' 2023-02-13T02:03:19,796 adding 'colossalai/fx/tracer/meta_patch/patched_module/normalization.py' 2023-02-13T02:03:19,799 adding 'colossalai/fx/tracer/meta_patch/patched_module/pooling.py' 2023-02-13T02:03:19,801 adding 'colossalai/fx/tracer/meta_patch/patched_module/rnn.py' 2023-02-13T02:03:19,804 adding 'colossalai/gemini/__init__.py' 2023-02-13T02:03:19,807 adding 'colossalai/gemini/gemini_context.py' 2023-02-13T02:03:19,810 adding 'colossalai/gemini/gemini_mgr.py' 2023-02-13T02:03:19,813 adding 'colossalai/gemini/placement_policy.py' 2023-02-13T02:03:19,817 adding 'colossalai/gemini/stateful_tensor.py' 2023-02-13T02:03:19,819 adding 'colossalai/gemini/stateful_tensor_mgr.py' 2023-02-13T02:03:19,822 adding 'colossalai/gemini/tensor_placement_policy.py' 2023-02-13T02:03:19,825 adding 'colossalai/gemini/tensor_utils.py' 2023-02-13T02:03:19,829 adding 'colossalai/gemini/chunk/__init__.py' 2023-02-13T02:03:19,835 adding 'colossalai/gemini/chunk/chunk.py' 2023-02-13T02:03:19,839 adding 'colossalai/gemini/chunk/manager.py' 2023-02-13T02:03:19,842 adding 'colossalai/gemini/chunk/search_utils.py' 2023-02-13T02:03:19,844 adding 'colossalai/gemini/chunk/utils.py' 2023-02-13T02:03:19,848 adding 'colossalai/gemini/memory_tracer/__init__.py' 2023-02-13T02:03:19,850 adding 'colossalai/gemini/memory_tracer/chunk_memstats_collector.py' 2023-02-13T02:03:19,853 adding 'colossalai/gemini/memory_tracer/memory_monitor.py' 2023-02-13T02:03:19,856 adding 'colossalai/gemini/memory_tracer/memory_stats.py' 2023-02-13T02:03:19,859 adding 'colossalai/gemini/memory_tracer/memstats_collector.py' 2023-02-13T02:03:19,861 adding 'colossalai/gemini/memory_tracer/param_runtime_order.py' 2023-02-13T02:03:19,864 adding 'colossalai/gemini/memory_tracer/runtime_mem_tracer.py' 2023-02-13T02:03:19,866 adding 'colossalai/gemini/memory_tracer/static_memstats_collector.py' 2023-02-13T02:03:19,869 adding 'colossalai/gemini/memory_tracer/utils.py' 2023-02-13T02:03:19,872 adding 'colossalai/gemini/ophooks/__init__.py' 2023-02-13T02:03:19,874 adding 'colossalai/gemini/ophooks/_shard_grad_ophook.py' 2023-02-13T02:03:19,877 adding 'colossalai/gemini/ophooks/_shard_param_ophook.py' 2023-02-13T02:03:19,879 adding 'colossalai/gemini/ophooks/runtime_mem_tracer_hook.py' 2023-02-13T02:03:19,882 adding 'colossalai/gemini/ophooks/utils.py' 2023-02-13T02:03:19,886 adding 'colossalai/gemini/paramhooks/__init__.py' 2023-02-13T02:03:19,888 adding 'colossalai/gemini/paramhooks/_param_hookmgr.py' 2023-02-13T02:03:19,891 adding 'colossalai/kernel/__init__.py' 2023-02-13T02:03:19,894 adding 'colossalai/kernel/cuda_native/__init__.py' 2023-02-13T02:03:19,899 adding 'colossalai/kernel/cuda_native/flash_attention.py' 2023-02-13T02:03:19,902 adding 'colossalai/kernel/cuda_native/layer_norm.py' 2023-02-13T02:03:19,906 adding 'colossalai/kernel/cuda_native/multihead_attention.py' 2023-02-13T02:03:19,909 adding 'colossalai/kernel/cuda_native/scaled_softmax.py' 2023-02-13T02:03:19,914 adding 'colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp' 2023-02-13T02:03:19,916 adding 'colossalai/kernel/cuda_native/csrc/compat.h' 2023-02-13T02:03:19,920 adding 'colossalai/kernel/cuda_native/csrc/cpu_adam.cpp' 2023-02-13T02:03:19,923 adding 'colossalai/kernel/cuda_native/csrc/cpu_adam.h' 2023-02-13T02:03:19,926 adding 'colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp' 2023-02-13T02:03:19,932 adding 'colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu' 2023-02-13T02:03:19,936 adding 'colossalai/kernel/cuda_native/csrc/moe_cuda.cpp' 2023-02-13T02:03:19,941 adding 'colossalai/kernel/cuda_native/csrc/moe_cuda_kernel.cu' 2023-02-13T02:03:19,945 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu' 2023-02-13T02:03:19,948 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_apply.cuh' 2023-02-13T02:03:19,952 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu' 2023-02-13T02:03:19,956 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu' 2023-02-13T02:03:19,959 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu' 2023-02-13T02:03:19,962 adding 'colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu' 2023-02-13T02:03:19,966 adding 'colossalai/kernel/cuda_native/csrc/multihead_attention_1d.cpp' 2023-02-13T02:03:19,970 adding 'colossalai/kernel/cuda_native/csrc/multihead_attention_1d.h' 2023-02-13T02:03:19,972 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.cpp' 2023-02-13T02:03:19,976 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax.h' 2023-02-13T02:03:19,979 adding 'colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu' 2023-02-13T02:03:19,982 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp' 2023-02-13T02:03:19,986 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.h' 2023-02-13T02:03:19,989 adding 'colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu' 2023-02-13T02:03:19,992 adding 'colossalai/kernel/cuda_native/csrc/type_shim.h' 2023-02-13T02:03:19,998 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/block_reduce.h' 2023-02-13T02:03:20,000 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/context.h' 2023-02-13T02:03:20,002 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cross_entropy_layer.h' 2023-02-13T02:03:20,005 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cublas_wrappers.h' 2023-02-13T02:03:20,007 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h' 2023-02-13T02:03:20,010 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/dropout.h' 2023-02-13T02:03:20,012 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/feed_forward.h' 2023-02-13T02:03:20,015 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/kernels.h' 2023-02-13T02:03:20,018 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/ls_cub.cuh' 2023-02-13T02:03:20,020 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/normalize_layer.h' 2023-02-13T02:03:20,022 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/softmax.h' 2023-02-13T02:03:20,025 adding 'colossalai/kernel/cuda_native/csrc/kernels/include/strided_batch_gemm.h' 2023-02-13T02:03:20,028 adding 'colossalai/kernel/jit/__init__.py' 2023-02-13T02:03:20,030 adding 'colossalai/kernel/jit/bias_dropout_add.py' 2023-02-13T02:03:20,033 adding 'colossalai/kernel/jit/bias_gelu.py' 2023-02-13T02:03:20,035 adding 'colossalai/kernel/jit/option.py' 2023-02-13T02:03:20,039 adding 'colossalai/kernel/op_builder/__init__.py' 2023-02-13T02:03:20,042 adding 'colossalai/kernel/op_builder/builder.py' 2023-02-13T02:03:20,045 adding 'colossalai/kernel/op_builder/cpu_adam.py' 2023-02-13T02:03:20,047 adding 'colossalai/kernel/op_builder/fused_optim.py' 2023-02-13T02:03:20,049 adding 'colossalai/kernel/op_builder/layernorm.py' 2023-02-13T02:03:20,051 adding 'colossalai/kernel/op_builder/moe.py' 2023-02-13T02:03:20,053 adding 'colossalai/kernel/op_builder/multi_head_attn.py' 2023-02-13T02:03:20,056 adding 'colossalai/kernel/op_builder/scaled_masked_softmax.py' 2023-02-13T02:03:20,058 adding 'colossalai/kernel/op_builder/scaled_upper_triangle_masked_softmax.py' 2023-02-13T02:03:20,060 adding 'colossalai/kernel/op_builder/utils.py' 2023-02-13T02:03:20,064 adding 'colossalai/logging/__init__.py' 2023-02-13T02:03:20,067 adding 'colossalai/logging/logger.py' 2023-02-13T02:03:20,070 adding 'colossalai/nn/__init__.py' 2023-02-13T02:03:20,073 adding 'colossalai/nn/init.py' 2023-02-13T02:03:20,076 adding 'colossalai/nn/_ops/__init__.py' 2023-02-13T02:03:20,080 adding 'colossalai/nn/_ops/_utils.py' 2023-02-13T02:03:20,082 adding 'colossalai/nn/_ops/addmm.py' 2023-02-13T02:03:20,085 adding 'colossalai/nn/_ops/batch_norm.py' 2023-02-13T02:03:20,088 adding 'colossalai/nn/_ops/element_wise.py' 2023-02-13T02:03:20,091 adding 'colossalai/nn/_ops/embedding.py' 2023-02-13T02:03:20,094 adding 'colossalai/nn/_ops/embedding_bag.py' 2023-02-13T02:03:20,096 adding 'colossalai/nn/_ops/layernorm.py' 2023-02-13T02:03:20,099 adding 'colossalai/nn/_ops/linear.py' 2023-02-13T02:03:20,101 adding 'colossalai/nn/_ops/loss.py' 2023-02-13T02:03:20,104 adding 'colossalai/nn/_ops/view.py' 2023-02-13T02:03:20,108 adding 'colossalai/nn/layer/__init__.py' 2023-02-13T02:03:20,110 adding 'colossalai/nn/layer/base_layer.py' 2023-02-13T02:03:20,113 adding 'colossalai/nn/layer/colossalai_layer/__init__.py' 2023-02-13T02:03:20,116 adding 'colossalai/nn/layer/colossalai_layer/_utils.py' 2023-02-13T02:03:20,118 adding 'colossalai/nn/layer/colossalai_layer/dropout.py' 2023-02-13T02:03:20,121 adding 'colossalai/nn/layer/colossalai_layer/embedding.py' 2023-02-13T02:03:20,124 adding 'colossalai/nn/layer/colossalai_layer/linear.py' 2023-02-13T02:03:20,126 adding 'colossalai/nn/layer/colossalai_layer/normalization.py' 2023-02-13T02:03:20,129 adding 'colossalai/nn/layer/moe/__init__.py' 2023-02-13T02:03:20,132 adding 'colossalai/nn/layer/moe/_operation.py' 2023-02-13T02:03:20,135 adding 'colossalai/nn/layer/moe/experts.py' 2023-02-13T02:03:20,139 adding 'colossalai/nn/layer/moe/layers.py' 2023-02-13T02:03:20,142 adding 'colossalai/nn/layer/moe/routers.py' 2023-02-13T02:03:20,145 adding 'colossalai/nn/layer/moe/utils.py' 2023-02-13T02:03:20,148 adding 'colossalai/nn/layer/parallel_1d/__init__.py' 2023-02-13T02:03:20,151 adding 'colossalai/nn/layer/parallel_1d/_operation.py' 2023-02-13T02:03:20,154 adding 'colossalai/nn/layer/parallel_1d/_utils.py' 2023-02-13T02:03:20,162 adding 'colossalai/nn/layer/parallel_1d/layers.py' 2023-02-13T02:03:20,166 adding 'colossalai/nn/layer/parallel_2d/__init__.py' 2023-02-13T02:03:20,172 adding 'colossalai/nn/layer/parallel_2d/_operation.py' 2023-02-13T02:03:20,175 adding 'colossalai/nn/layer/parallel_2d/_utils.py' 2023-02-13T02:03:20,183 adding 'colossalai/nn/layer/parallel_2d/layers.py' 2023-02-13T02:03:20,187 adding 'colossalai/nn/layer/parallel_2p5d/__init__.py' 2023-02-13T02:03:20,194 adding 'colossalai/nn/layer/parallel_2p5d/_operation.py' 2023-02-13T02:03:20,197 adding 'colossalai/nn/layer/parallel_2p5d/_utils.py' 2023-02-13T02:03:20,205 adding 'colossalai/nn/layer/parallel_2p5d/layers.py' 2023-02-13T02:03:20,210 adding 'colossalai/nn/layer/parallel_3d/__init__.py' 2023-02-13T02:03:20,214 adding 'colossalai/nn/layer/parallel_3d/_operation.py' 2023-02-13T02:03:20,217 adding 'colossalai/nn/layer/parallel_3d/_utils.py' 2023-02-13T02:03:20,225 adding 'colossalai/nn/layer/parallel_3d/layers.py' 2023-02-13T02:03:20,229 adding 'colossalai/nn/layer/parallel_sequence/__init__.py' 2023-02-13T02:03:20,232 adding 'colossalai/nn/layer/parallel_sequence/_operation.py' 2023-02-13T02:03:20,234 adding 'colossalai/nn/layer/parallel_sequence/_utils.py' 2023-02-13T02:03:20,238 adding 'colossalai/nn/layer/parallel_sequence/layers.py' 2023-02-13T02:03:20,241 adding 'colossalai/nn/layer/utils/__init__.py' 2023-02-13T02:03:20,243 adding 'colossalai/nn/layer/utils/common.py' 2023-02-13T02:03:20,247 adding 'colossalai/nn/layer/vanilla/__init__.py' 2023-02-13T02:03:20,251 adding 'colossalai/nn/layer/vanilla/layers.py' 2023-02-13T02:03:20,254 adding 'colossalai/nn/layer/wrapper/__init__.py' 2023-02-13T02:03:20,256 adding 'colossalai/nn/layer/wrapper/pipeline_wrapper.py' 2023-02-13T02:03:20,260 adding 'colossalai/nn/loss/__init__.py' 2023-02-13T02:03:20,262 adding 'colossalai/nn/loss/loss_1d.py' 2023-02-13T02:03:20,266 adding 'colossalai/nn/loss/loss_2d.py' 2023-02-13T02:03:20,268 adding 'colossalai/nn/loss/loss_2p5d.py' 2023-02-13T02:03:20,271 adding 'colossalai/nn/loss/loss_3d.py' 2023-02-13T02:03:20,274 adding 'colossalai/nn/loss/loss_moe.py' 2023-02-13T02:03:20,277 adding 'colossalai/nn/lr_scheduler/__init__.py' 2023-02-13T02:03:20,280 adding 'colossalai/nn/lr_scheduler/cosine.py' 2023-02-13T02:03:20,283 adding 'colossalai/nn/lr_scheduler/delayed.py' 2023-02-13T02:03:20,286 adding 'colossalai/nn/lr_scheduler/linear.py' 2023-02-13T02:03:20,288 adding 'colossalai/nn/lr_scheduler/multistep.py' 2023-02-13T02:03:20,291 adding 'colossalai/nn/lr_scheduler/onecycle.py' 2023-02-13T02:03:20,293 adding 'colossalai/nn/lr_scheduler/poly.py' 2023-02-13T02:03:20,296 adding 'colossalai/nn/lr_scheduler/torch.py' 2023-02-13T02:03:20,299 adding 'colossalai/nn/metric/__init__.py' 2023-02-13T02:03:20,301 adding 'colossalai/nn/metric/_utils.py' 2023-02-13T02:03:20,304 adding 'colossalai/nn/metric/accuracy_2d.py' 2023-02-13T02:03:20,306 adding 'colossalai/nn/metric/accuracy_2p5d.py' 2023-02-13T02:03:20,308 adding 'colossalai/nn/metric/accuracy_3d.py' 2023-02-13T02:03:20,311 adding 'colossalai/nn/optimizer/__init__.py' 2023-02-13T02:03:20,314 adding 'colossalai/nn/optimizer/colossalai_optimizer.py' 2023-02-13T02:03:20,317 adding 'colossalai/nn/optimizer/cpu_adam.py' 2023-02-13T02:03:20,320 adding 'colossalai/nn/optimizer/fused_adam.py' 2023-02-13T02:03:20,324 adding 'colossalai/nn/optimizer/fused_lamb.py' 2023-02-13T02:03:20,327 adding 'colossalai/nn/optimizer/fused_sgd.py' 2023-02-13T02:03:20,329 adding 'colossalai/nn/optimizer/gemini_optimizer.py' 2023-02-13T02:03:20,332 adding 'colossalai/nn/optimizer/hybrid_adam.py' 2023-02-13T02:03:20,335 adding 'colossalai/nn/optimizer/lamb.py' 2023-02-13T02:03:20,338 adding 'colossalai/nn/optimizer/lars.py' 2023-02-13T02:03:20,341 adding 'colossalai/nn/optimizer/nvme_optimizer.py' 2023-02-13T02:03:20,346 adding 'colossalai/nn/optimizer/zero_optimizer.py' 2023-02-13T02:03:20,350 adding 'colossalai/nn/parallel/__init__.py' 2023-02-13T02:03:20,359 adding 'colossalai/nn/parallel/data_parallel.py' 2023-02-13T02:03:20,362 adding 'colossalai/nn/parallel/gemini_parallel.py' 2023-02-13T02:03:20,365 adding 'colossalai/nn/parallel/reducer.py' 2023-02-13T02:03:20,367 adding 'colossalai/nn/parallel/utils.py' 2023-02-13T02:03:20,370 adding 'colossalai/nn/parallel/zero_wrapper.py' 2023-02-13T02:03:20,374 adding 'colossalai/nn/parallel/layers/__init__.py' 2023-02-13T02:03:20,376 adding 'colossalai/nn/parallel/layers/colo_module.py' 2023-02-13T02:03:20,379 adding 'colossalai/nn/parallel/layers/embedding.py' 2023-02-13T02:03:20,381 adding 'colossalai/nn/parallel/layers/linear.py' 2023-02-13T02:03:20,384 adding 'colossalai/nn/parallel/layers/module_utils.py' 2023-02-13T02:03:20,387 adding 'colossalai/nn/parallel/layers/cache_embedding/__init__.py' 2023-02-13T02:03:20,390 adding 'colossalai/nn/parallel/layers/cache_embedding/base_embedding.py' 2023-02-13T02:03:20,397 adding 'colossalai/nn/parallel/layers/cache_embedding/cache_mgr.py' 2023-02-13T02:03:20,401 adding 'colossalai/nn/parallel/layers/cache_embedding/cached_embedding.py' 2023-02-13T02:03:20,404 adding 'colossalai/nn/parallel/layers/cache_embedding/copyer.py' 2023-02-13T02:03:20,406 adding 'colossalai/nn/parallel/layers/cache_embedding/embedding_config.py' 2023-02-13T02:03:20,409 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding.py' 2023-02-13T02:03:20,412 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise.py' 2023-02-13T02:03:20,416 adding 'colossalai/nn/parallel/layers/cache_embedding/parallel_cached_embedding_tablewise_split_cache.py' 2023-02-13T02:03:20,419 adding 'colossalai/pipeline/__init__.py' 2023-02-13T02:03:20,421 adding 'colossalai/pipeline/layer_spec.py' 2023-02-13T02:03:20,425 adding 'colossalai/pipeline/pipelinable.py' 2023-02-13T02:03:20,428 adding 'colossalai/pipeline/pipeline_process_group.py' 2023-02-13T02:03:20,432 adding 'colossalai/pipeline/utils.py' 2023-02-13T02:03:20,436 adding 'colossalai/pipeline/middleware/__init__.py' 2023-02-13T02:03:20,439 adding 'colossalai/pipeline/middleware/topo.py' 2023-02-13T02:03:20,442 adding 'colossalai/pipeline/middleware/adaptor/__init__.py' 2023-02-13T02:03:20,445 adding 'colossalai/pipeline/middleware/adaptor/fx.py' 2023-02-13T02:03:20,448 adding 'colossalai/pipeline/rpc/__init__.py' 2023-02-13T02:03:20,463 adding 'colossalai/pipeline/rpc/_pipeline_base.py' 2023-02-13T02:03:20,468 adding 'colossalai/pipeline/rpc/_pipeline_schedule.py' 2023-02-13T02:03:20,471 adding 'colossalai/pipeline/rpc/utils.py' 2023-02-13T02:03:20,475 adding 'colossalai/registry/__init__.py' 2023-02-13T02:03:20,478 adding 'colossalai/registry/registry.py' 2023-02-13T02:03:20,481 adding 'colossalai/tensor/__init__.py' 2023-02-13T02:03:20,485 adding 'colossalai/tensor/colo_parameter.py' 2023-02-13T02:03:20,489 adding 'colossalai/tensor/colo_tensor.py' 2023-02-13T02:03:20,495 adding 'colossalai/tensor/comm_spec.py' 2023-02-13T02:03:20,498 adding 'colossalai/tensor/compute_spec.py' 2023-02-13T02:03:20,500 adding 'colossalai/tensor/const.py' 2023-02-13T02:03:20,503 adding 'colossalai/tensor/dist_spec_mgr.py' 2023-02-13T02:03:20,506 adding 'colossalai/tensor/distspec.py' 2023-02-13T02:03:20,508 adding 'colossalai/tensor/op_wrapper.py' 2023-02-13T02:03:20,511 adding 'colossalai/tensor/param_op_hook.py' 2023-02-13T02:03:20,515 adding 'colossalai/tensor/process_group.py' 2023-02-13T02:03:20,523 adding 'colossalai/tensor/shape_consistency.py' 2023-02-13T02:03:20,528 adding 'colossalai/tensor/sharding_spec.py' 2023-02-13T02:03:20,530 adding 'colossalai/tensor/tensor_spec.py' 2023-02-13T02:03:20,534 adding 'colossalai/tensor/utils.py' 2023-02-13T02:03:20,537 adding 'colossalai/testing/__init__.py' 2023-02-13T02:03:20,540 adding 'colossalai/testing/comparison.py' 2023-02-13T02:03:20,542 adding 'colossalai/testing/pytest_wrapper.py' 2023-02-13T02:03:20,544 adding 'colossalai/testing/random.py' 2023-02-13T02:03:20,547 adding 'colossalai/testing/utils.py' 2023-02-13T02:03:20,550 adding 'colossalai/trainer/__init__.py' 2023-02-13T02:03:20,555 adding 'colossalai/trainer/_trainer.py' 2023-02-13T02:03:20,558 adding 'colossalai/trainer/hooks/__init__.py' 2023-02-13T02:03:20,561 adding 'colossalai/trainer/hooks/_base_hook.py' 2023-02-13T02:03:20,563 adding 'colossalai/trainer/hooks/_checkpoint_hook.py' 2023-02-13T02:03:20,566 adding 'colossalai/trainer/hooks/_commons_.py' 2023-02-13T02:03:20,569 adding 'colossalai/trainer/hooks/_log_hook.py' 2023-02-13T02:03:20,572 adding 'colossalai/trainer/hooks/_lr_scheduler_hook.py' 2023-02-13T02:03:20,576 adding 'colossalai/trainer/hooks/_metric_hook.py' 2023-02-13T02:03:20,580 adding 'colossalai/utils/__init__.py' 2023-02-13T02:03:20,584 adding 'colossalai/utils/activation_checkpoint.py' 2023-02-13T02:03:20,588 adding 'colossalai/utils/checkpointing.py' 2023-02-13T02:03:20,593 adding 'colossalai/utils/common.py' 2023-02-13T02:03:20,596 adding 'colossalai/utils/cuda.py' 2023-02-13T02:03:20,599 adding 'colossalai/utils/memory.py' 2023-02-13T02:03:20,602 adding 'colossalai/utils/moe.py' 2023-02-13T02:03:20,604 adding 'colossalai/utils/timer.py' 2023-02-13T02:03:20,608 adding 'colossalai/utils/checkpoint/__init__.py' 2023-02-13T02:03:20,611 adding 'colossalai/utils/checkpoint/module_checkpoint.py' 2023-02-13T02:03:20,613 adding 'colossalai/utils/checkpoint/utils.py' 2023-02-13T02:03:20,617 adding 'colossalai/utils/checkpoint_io/__init__.py' 2023-02-13T02:03:20,619 adding 'colossalai/utils/checkpoint_io/backend.py' 2023-02-13T02:03:20,621 adding 'colossalai/utils/checkpoint_io/constant.py' 2023-02-13T02:03:20,624 adding 'colossalai/utils/checkpoint_io/convertor.py' 2023-02-13T02:03:20,627 adding 'colossalai/utils/checkpoint_io/distributed.py' 2023-02-13T02:03:20,631 adding 'colossalai/utils/checkpoint_io/io.py' 2023-02-13T02:03:20,633 adding 'colossalai/utils/checkpoint_io/meta.py' 2023-02-13T02:03:20,636 adding 'colossalai/utils/checkpoint_io/reader.py' 2023-02-13T02:03:20,640 adding 'colossalai/utils/checkpoint_io/utils.py' 2023-02-13T02:03:20,642 adding 'colossalai/utils/checkpoint_io/writer.py' 2023-02-13T02:03:20,646 adding 'colossalai/utils/data_sampler/__init__.py' 2023-02-13T02:03:20,648 adding 'colossalai/utils/data_sampler/base_sampler.py' 2023-02-13T02:03:20,651 adding 'colossalai/utils/data_sampler/data_parallel_sampler.py' 2023-02-13T02:03:20,654 adding 'colossalai/utils/model/__init__.py' 2023-02-13T02:03:20,658 adding 'colossalai/utils/model/colo_init_context.py' 2023-02-13T02:03:20,662 adding 'colossalai/utils/model/experimental.py' 2023-02-13T02:03:20,666 adding 'colossalai/utils/model/lazy_init_context.py' 2023-02-13T02:03:20,669 adding 'colossalai/utils/model/utils.py' 2023-02-13T02:03:20,672 adding 'colossalai/utils/multi_tensor_apply/__init__.py' 2023-02-13T02:03:20,674 adding 'colossalai/utils/multi_tensor_apply/multi_tensor_apply.py' 2023-02-13T02:03:20,678 adding 'colossalai/utils/profiler/__init__.py' 2023-02-13T02:03:20,680 adding 'colossalai/utils/profiler/extention.py' 2023-02-13T02:03:20,683 adding 'colossalai/utils/profiler/profiler.py' 2023-02-13T02:03:20,686 adding 'colossalai/utils/profiler/stateful_tensor_mem_extention.py' 2023-02-13T02:03:20,689 adding 'colossalai/utils/profiler/legacy/__init__.py' 2023-02-13T02:03:20,693 adding 'colossalai/utils/profiler/legacy/comm_profiler.py' 2023-02-13T02:03:20,696 adding 'colossalai/utils/profiler/legacy/pcie_profiler.py' 2023-02-13T02:03:20,699 adding 'colossalai/utils/profiler/legacy/prof_utils.py' 2023-02-13T02:03:20,702 adding 'colossalai/utils/rank_recorder/__init__.py' 2023-02-13T02:03:20,705 adding 'colossalai/utils/rank_recorder/rank_recorder.py' 2023-02-13T02:03:20,708 adding 'colossalai/utils/tensor_detector/__init__.py' 2023-02-13T02:03:20,711 adding 'colossalai/utils/tensor_detector/tensor_detector.py' 2023-02-13T02:03:20,715 adding 'colossalai/zero/__init__.py' 2023-02-13T02:03:20,718 adding 'colossalai/zero/init_ctx/__init__.py' 2023-02-13T02:03:20,722 adding 'colossalai/zero/init_ctx/init_context.py' 2023-02-13T02:03:20,726 adding 'colossalai/zero/shard_utils/__init__.py' 2023-02-13T02:03:20,728 adding 'colossalai/zero/shard_utils/base_shard_strategy.py' 2023-02-13T02:03:20,731 adding 'colossalai/zero/shard_utils/bucket_tensor_shard_strategy.py' 2023-02-13T02:03:20,733 adding 'colossalai/zero/shard_utils/commons.py' 2023-02-13T02:03:20,735 adding 'colossalai/zero/shard_utils/tensor_shard_strategy.py' 2023-02-13T02:03:20,739 adding 'colossalai/zero/sharded_model/__init__.py' 2023-02-13T02:03:20,745 adding 'colossalai/zero/sharded_model/_utils.py' 2023-02-13T02:03:20,748 adding 'colossalai/zero/sharded_model/reduce_scatter.py' 2023-02-13T02:03:20,756 adding 'colossalai/zero/sharded_model/sharded_model_v2.py' 2023-02-13T02:03:20,759 adding 'colossalai/zero/sharded_model/utils.py' 2023-02-13T02:03:20,762 adding 'colossalai/zero/sharded_optim/__init__.py' 2023-02-13T02:03:20,766 adding 'colossalai/zero/sharded_optim/_utils.py' 2023-02-13T02:03:20,773 adding 'colossalai/zero/sharded_optim/low_level_optim.py' 2023-02-13T02:03:20,779 adding 'colossalai/zero/sharded_optim/sharded_optim_v2.py' 2023-02-13T02:03:20,782 adding 'colossalai/zero/sharded_optim/bookkeeping/__init__.py' 2023-02-13T02:03:20,784 adding 'colossalai/zero/sharded_optim/bookkeeping/base_store.py' 2023-02-13T02:03:20,787 adding 'colossalai/zero/sharded_optim/bookkeeping/bucket_store.py' 2023-02-13T02:03:20,789 adding 'colossalai/zero/sharded_optim/bookkeeping/gradient_store.py' 2023-02-13T02:03:20,792 adding 'colossalai/zero/sharded_optim/bookkeeping/parameter_store.py' 2023-02-13T02:03:20,794 adding 'colossalai/zero/sharded_optim/bookkeeping/tensor_bucket.py' 2023-02-13T02:03:20,797 adding 'colossalai/zero/sharded_param/__init__.py' 2023-02-13T02:03:20,800 adding 'colossalai/zero/sharded_param/sharded_param.py' 2023-02-13T02:03:20,802 adding 'colossalai/zero/sharded_param/sharded_tensor.py' 2023-02-13T02:03:20,805 adding 'colossalai/zero/utils/__init__.py' 2023-02-13T02:03:20,808 adding 'colossalai/zero/utils/gemini_hook.py' 2023-02-13T02:03:20,810 adding 'colossalai/zero/utils/zero_hook.py' 2023-02-13T02:03:20,817 adding 'op_builder/__init__.py' 2023-02-13T02:03:20,820 adding 'op_builder/builder.py' 2023-02-13T02:03:20,822 adding 'op_builder/cpu_adam.py' 2023-02-13T02:03:20,825 adding 'op_builder/fused_optim.py' 2023-02-13T02:03:20,827 adding 'op_builder/layernorm.py' 2023-02-13T02:03:20,829 adding 'op_builder/moe.py' 2023-02-13T02:03:20,831 adding 'op_builder/multi_head_attn.py' 2023-02-13T02:03:20,834 adding 'op_builder/scaled_masked_softmax.py' 2023-02-13T02:03:20,836 adding 'op_builder/scaled_upper_triangle_masked_softmax.py' 2023-02-13T02:03:20,839 adding 'op_builder/utils.py' 2023-02-13T02:03:20,843 adding 'colossalai-0.2.3.dist-info/LICENSE' 2023-02-13T02:03:20,847 adding 'colossalai-0.2.3.dist-info/METADATA' 2023-02-13T02:03:20,850 adding 'colossalai-0.2.3.dist-info/WHEEL' 2023-02-13T02:03:20,851 adding 'colossalai-0.2.3.dist-info/entry_points.txt' 2023-02-13T02:03:20,853 adding 'colossalai-0.2.3.dist-info/top_level.txt' 2023-02-13T02:03:20,879 adding 'colossalai-0.2.3.dist-info/RECORD' 2023-02-13T02:03:20,936 removing build/bdist.linux-armv7l/wheel 2023-02-13T02:03:21,511 Building wheel for colossalai (setup.py): finished with status 'done' 2023-02-13T02:03:21,546 Created wheel for colossalai: filename=colossalai-0.2.3-py3-none-any.whl size=926118 sha256=e5c1d7c77bd0eb9c0b3cea5c33050a759bb29fcd128ef0f6459bec9e69f4969c 2023-02-13T02:03:21,548 Stored in directory: /tmp/pip-ephem-wheel-cache-k7hlq7wg/wheels/7b/81/46/2fa6d311906138d2c96d61cbb6138ccfa6378aa6905b24d128 2023-02-13T02:03:21,627 Successfully built colossalai 2023-02-13T02:03:21,701 Removed build tracker: '/tmp/pip-build-tracker-m8sjly61'