2024-12-06T09:55:25,908 Created temporary directory: /tmp/pip-build-tracker-201tn4cd 2024-12-06T09:55:25,910 Initialized build tracking at /tmp/pip-build-tracker-201tn4cd 2024-12-06T09:55:25,910 Created build tracker: /tmp/pip-build-tracker-201tn4cd 2024-12-06T09:55:25,911 Entered build tracker: /tmp/pip-build-tracker-201tn4cd 2024-12-06T09:55:25,912 Created temporary directory: /tmp/pip-wheel-czzzp227 2024-12-06T09:55:25,915 Created temporary directory: /tmp/pip-ephem-wheel-cache-ms59vp62 2024-12-06T09:55:25,959 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2024-12-06T09:55:25,962 2 location(s) to search for versions of py-data-juicer: 2024-12-06T09:55:25,962 * https://pypi.org/simple/py-data-juicer/ 2024-12-06T09:55:25,962 * https://www.piwheels.org/simple/py-data-juicer/ 2024-12-06T09:55:25,963 Fetching project page and analyzing links: https://pypi.org/simple/py-data-juicer/ 2024-12-06T09:55:25,963 Getting page https://pypi.org/simple/py-data-juicer/ 2024-12-06T09:55:25,965 Found index url https://pypi.org/simple/ 2024-12-06T09:55:26,225 Fetched page https://pypi.org/simple/py-data-juicer/ as application/vnd.pypi.simple.v1+json 2024-12-06T09:55:26,228 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/e3/d3/d0724d922e7c55a0664485fdf642124bdcd801df2697e29c9463c4360958/py_data_juicer-0.1.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-06T09:55:26,229 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/fd/c6/e1428310bf534319ddc2e362c6eee21985f097e70cedc0e9cd3e914de826/py_data_juicer-0.1.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-06T09:55:26,229 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/67/70/731e349d2a92bf59a767230b956665dd27ea24a7a948d4a1710154d77a24/py_data_juicer-0.1.2-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-06T09:55:26,230 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/66/08/f584efbdf8277a061ce73c9befc33fec813cbd08020761af2380dd331692/py_data_juicer-0.1.3-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-06T09:55:26,230 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/69/20/9862dfe7a94f10caa0e6387834d1265420613345c7adadd41a3ced7230e8/py_data_juicer-0.2.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-06T09:55:26,231 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/05/34/33fb401d350f1a66cdaefc224419f9d01484722829f5b0c71ef59a8365a4/py_data_juicer-1.0.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-06T09:55:26,232 Found link https://files.pythonhosted.org/packages/49/e9/cfb994255490c36554048e0b3956858e61f5a75ffacfde20849f31f04ef8/py_data_juicer-1.0.0.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.0 2024-12-06T09:55:26,233 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/27/37/150e2198f14349fdd6b647a63feaede809e21dbfc4617aba487f18049828/py_data_juicer-1.0.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-06T09:55:26,233 Found link https://files.pythonhosted.org/packages/4a/60/dadcbe4337a76d8f98022b60d142ac31073d568c9e06a5461b3014d16fc6/py_data_juicer-1.0.1.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.1 2024-12-06T09:55:26,234 Fetching project page and analyzing links: https://www.piwheels.org/simple/py-data-juicer/ 2024-12-06T09:55:26,235 Getting page https://www.piwheels.org/simple/py-data-juicer/ 2024-12-06T09:55:26,236 Found index url https://www.piwheels.org/simple/ 2024-12-06T09:55:26,404 Fetched page https://www.piwheels.org/simple/py-data-juicer/ as text/html 2024-12-06T09:55:26,405 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.0-py3-none-any.whl#sha256=7cd007456933447474ec9faf5357b741a63e7a0c0dc6230e40bf4498eb1ef554 (from https://www.piwheels.org/simple/py-data-juicer/) 2024-12-06T09:55:26,406 Skipping link: not a file: https://www.piwheels.org/simple/py-data-juicer/ 2024-12-06T09:55:26,407 Skipping link: not a file: https://pypi.org/simple/py-data-juicer/ 2024-12-06T09:55:26,426 Given no hashes to check 1 links for project 'py-data-juicer': discarding no candidates 2024-12-06T09:55:26,428 Collecting py-data-juicer==1.0.1 2024-12-06T09:55:26,431 Created temporary directory: /tmp/pip-unpack-_2zb39i3 2024-12-06T09:55:26,648 Downloading py_data_juicer-1.0.1.tar.gz (226 kB) 2024-12-06T09:55:27,118 Added py-data-juicer==1.0.1 from https://files.pythonhosted.org/packages/4a/60/dadcbe4337a76d8f98022b60d142ac31073d568c9e06a5461b3014d16fc6/py_data_juicer-1.0.1.tar.gz to build tracker '/tmp/pip-build-tracker-201tn4cd' 2024-12-06T09:55:27,120 Running setup.py (path:/tmp/pip-wheel-czzzp227/py-data-juicer_9bc22da256f44222a8c56a7e964e52ce/setup.py) egg_info for package py-data-juicer 2024-12-06T09:55:27,121 Created temporary directory: /tmp/pip-pip-egg-info-2huh9n33 2024-12-06T09:55:27,121 Preparing metadata (setup.py): started 2024-12-06T09:55:27,123 Running command python setup.py egg_info 2024-12-06T09:55:27,608 WARNING:root:target file does not exist: environments/minimal_requires.txt 2024-12-06T09:55:27,608 WARNING:root:target file does not exist: environments/science_requires.txt 2024-12-06T09:55:27,609 WARNING:root:target file does not exist: environments/dist_requires.txt 2024-12-06T09:55:27,610 WARNING:root:target file does not exist: environments/dev_requires.txt 2024-12-06T09:55:27,610 WARNING:root:target file does not exist: environments/preprocess_requires.txt 2024-12-06T09:55:27,611 WARNING:root:target file does not exist: environments/quality_classifier_requires.txt 2024-12-06T09:55:27,611 WARNING:root:target file does not exist: environments/sandbox_requires.txt 2024-12-06T09:55:28,223 INFO:root:running egg_info 2024-12-06T09:55:28,252 INFO:root:creating /tmp/pip-pip-egg-info-2huh9n33/py_data_juicer.egg-info 2024-12-06T09:55:28,253 INFO:root:writing /tmp/pip-pip-egg-info-2huh9n33/py_data_juicer.egg-info/PKG-INFO 2024-12-06T09:55:28,257 INFO:root:writing dependency_links to /tmp/pip-pip-egg-info-2huh9n33/py_data_juicer.egg-info/dependency_links.txt 2024-12-06T09:55:28,259 INFO:root:writing entry points to /tmp/pip-pip-egg-info-2huh9n33/py_data_juicer.egg-info/entry_points.txt 2024-12-06T09:55:28,261 INFO:root:writing requirements to /tmp/pip-pip-egg-info-2huh9n33/py_data_juicer.egg-info/requires.txt 2024-12-06T09:55:28,262 INFO:root:writing top-level names to /tmp/pip-pip-egg-info-2huh9n33/py_data_juicer.egg-info/top_level.txt 2024-12-06T09:55:28,264 INFO:root:writing manifest file '/tmp/pip-pip-egg-info-2huh9n33/py_data_juicer.egg-info/SOURCES.txt' 2024-12-06T09:55:28,368 INFO:root:reading manifest file '/tmp/pip-pip-egg-info-2huh9n33/py_data_juicer.egg-info/SOURCES.txt' 2024-12-06T09:55:28,370 INFO:root:adding license file 'LICENSE' 2024-12-06T09:55:28,378 INFO:root:writing manifest file '/tmp/pip-pip-egg-info-2huh9n33/py_data_juicer.egg-info/SOURCES.txt' 2024-12-06T09:55:28,469 Preparing metadata (setup.py): finished with status 'done' 2024-12-06T09:55:28,475 Source in /tmp/pip-wheel-czzzp227/py-data-juicer_9bc22da256f44222a8c56a7e964e52ce has version 1.0.1, which satisfies requirement py-data-juicer==1.0.1 from https://files.pythonhosted.org/packages/4a/60/dadcbe4337a76d8f98022b60d142ac31073d568c9e06a5461b3014d16fc6/py_data_juicer-1.0.1.tar.gz 2024-12-06T09:55:28,476 Removed py-data-juicer==1.0.1 from https://files.pythonhosted.org/packages/4a/60/dadcbe4337a76d8f98022b60d142ac31073d568c9e06a5461b3014d16fc6/py_data_juicer-1.0.1.tar.gz from build tracker '/tmp/pip-build-tracker-201tn4cd' 2024-12-06T09:55:28,483 Created temporary directory: /tmp/pip-unpack-_k_0lydq 2024-12-06T09:55:28,485 Created temporary directory: /tmp/pip-unpack-z082s2h8 2024-12-06T09:55:28,485 Building wheels for collected packages: py-data-juicer 2024-12-06T09:55:28,489 Created temporary directory: /tmp/pip-wheel-mmgjowf0 2024-12-06T09:55:28,490 Building wheel for py-data-juicer (setup.py): started 2024-12-06T09:55:28,491 Destination directory: /tmp/pip-wheel-mmgjowf0 2024-12-06T09:55:28,492 Running command python setup.py bdist_wheel 2024-12-06T09:55:28,943 WARNING:root:target file does not exist: environments/minimal_requires.txt 2024-12-06T09:55:28,943 WARNING:root:target file does not exist: environments/science_requires.txt 2024-12-06T09:55:28,943 WARNING:root:target file does not exist: environments/dist_requires.txt 2024-12-06T09:55:28,944 WARNING:root:target file does not exist: environments/dev_requires.txt 2024-12-06T09:55:28,945 WARNING:root:target file does not exist: environments/preprocess_requires.txt 2024-12-06T09:55:28,945 WARNING:root:target file does not exist: environments/quality_classifier_requires.txt 2024-12-06T09:55:28,945 WARNING:root:target file does not exist: environments/sandbox_requires.txt 2024-12-06T09:55:29,522 INFO:root:running bdist_wheel 2024-12-06T09:55:29,669 INFO:root:running build 2024-12-06T09:55:29,670 INFO:root:running build_py 2024-12-06T09:55:29,702 INFO:root:creating build/lib/data_juicer 2024-12-06T09:55:29,705 INFO:root:copying data_juicer/__init__.py -> build/lib/data_juicer 2024-12-06T09:55:29,708 INFO:root:creating build/lib/data_juicer/format 2024-12-06T09:55:29,709 INFO:root:copying data_juicer/format/load.py -> build/lib/data_juicer/format 2024-12-06T09:55:29,712 INFO:root:copying data_juicer/format/empty_formatter.py -> build/lib/data_juicer/format 2024-12-06T09:55:29,714 INFO:root:copying data_juicer/format/formatter.py -> build/lib/data_juicer/format 2024-12-06T09:55:29,717 INFO:root:copying data_juicer/format/tsv_formatter.py -> build/lib/data_juicer/format 2024-12-06T09:55:29,719 INFO:root:copying data_juicer/format/csv_formatter.py -> build/lib/data_juicer/format 2024-12-06T09:55:29,721 INFO:root:copying data_juicer/format/mixture_formatter.py -> build/lib/data_juicer/format 2024-12-06T09:55:29,724 INFO:root:copying data_juicer/format/parquet_formatter.py -> build/lib/data_juicer/format 2024-12-06T09:55:29,726 INFO:root:copying data_juicer/format/json_formatter.py -> build/lib/data_juicer/format 2024-12-06T09:55:29,728 INFO:root:copying data_juicer/format/__init__.py -> build/lib/data_juicer/format 2024-12-06T09:55:29,730 INFO:root:copying data_juicer/format/text_formatter.py -> build/lib/data_juicer/format 2024-12-06T09:55:29,733 INFO:root:creating build/lib/data_juicer/analysis 2024-12-06T09:55:29,734 INFO:root:copying data_juicer/analysis/collector.py -> build/lib/data_juicer/analysis 2024-12-06T09:55:29,736 INFO:root:copying data_juicer/analysis/draw.py -> build/lib/data_juicer/analysis 2024-12-06T09:55:29,738 INFO:root:copying data_juicer/analysis/diversity_analysis.py -> build/lib/data_juicer/analysis 2024-12-06T09:55:29,740 INFO:root:copying data_juicer/analysis/overall_analysis.py -> build/lib/data_juicer/analysis 2024-12-06T09:55:29,742 INFO:root:copying data_juicer/analysis/__init__.py -> build/lib/data_juicer/analysis 2024-12-06T09:55:29,744 INFO:root:copying data_juicer/analysis/measure.py -> build/lib/data_juicer/analysis 2024-12-06T09:55:29,746 INFO:root:copying data_juicer/analysis/column_wise_analysis.py -> build/lib/data_juicer/analysis 2024-12-06T09:55:29,749 INFO:root:creating build/lib/data_juicer/utils 2024-12-06T09:55:29,750 INFO:root:copying data_juicer/utils/registry.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,752 INFO:root:copying data_juicer/utils/availability_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,754 INFO:root:copying data_juicer/utils/logger_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,764 INFO:root:copying data_juicer/utils/resource_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,766 INFO:root:copying data_juicer/utils/constant.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,768 INFO:root:copying data_juicer/utils/__init__.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,770 INFO:root:copying data_juicer/utils/ckpt_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,772 INFO:root:copying data_juicer/utils/file_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,774 INFO:root:copying data_juicer/utils/unittest_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,777 INFO:root:copying data_juicer/utils/asset_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,778 INFO:root:copying data_juicer/utils/model_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,782 INFO:root:copying data_juicer/utils/auto_install_mapping.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,784 INFO:root:copying data_juicer/utils/mm_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,786 INFO:root:copying data_juicer/utils/fingerprint_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,789 INFO:root:copying data_juicer/utils/auto_install_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,790 INFO:root:copying data_juicer/utils/compress.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,793 INFO:root:copying data_juicer/utils/process_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,795 INFO:root:copying data_juicer/utils/common_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,797 INFO:root:copying data_juicer/utils/cache_utils.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,798 INFO:root:copying data_juicer/utils/lazy_loader.py -> build/lib/data_juicer/utils 2024-12-06T09:55:29,801 INFO:root:creating build/lib/data_juicer/config 2024-12-06T09:55:29,802 INFO:root:copying data_juicer/config/config.py -> build/lib/data_juicer/config 2024-12-06T09:55:29,805 INFO:root:copying data_juicer/config/__init__.py -> build/lib/data_juicer/config 2024-12-06T09:55:29,808 INFO:root:creating build/lib/data_juicer/ops 2024-12-06T09:55:29,809 INFO:root:copying data_juicer/ops/load.py -> build/lib/data_juicer/ops 2024-12-06T09:55:29,811 INFO:root:copying data_juicer/ops/__init__.py -> build/lib/data_juicer/ops 2024-12-06T09:55:29,813 INFO:root:copying data_juicer/ops/op_fusion.py -> build/lib/data_juicer/ops 2024-12-06T09:55:29,815 INFO:root:copying data_juicer/ops/base_op.py -> build/lib/data_juicer/ops 2024-12-06T09:55:29,818 INFO:root:creating build/lib/data_juicer/core 2024-12-06T09:55:29,819 INFO:root:copying data_juicer/core/executor.py -> build/lib/data_juicer/core 2024-12-06T09:55:29,822 INFO:root:copying data_juicer/core/tracer.py -> build/lib/data_juicer/core 2024-12-06T09:55:29,824 INFO:root:copying data_juicer/core/data.py -> build/lib/data_juicer/core 2024-12-06T09:55:29,827 INFO:root:copying data_juicer/core/exporter.py -> build/lib/data_juicer/core 2024-12-06T09:55:29,829 INFO:root:copying data_juicer/core/__init__.py -> build/lib/data_juicer/core 2024-12-06T09:55:29,831 INFO:root:copying data_juicer/core/ray_executor.py -> build/lib/data_juicer/core 2024-12-06T09:55:29,833 INFO:root:copying data_juicer/core/monitor.py -> build/lib/data_juicer/core 2024-12-06T09:55:29,835 INFO:root:copying data_juicer/core/adapter.py -> build/lib/data_juicer/core 2024-12-06T09:55:29,838 INFO:root:copying data_juicer/core/ray_data.py -> build/lib/data_juicer/core 2024-12-06T09:55:29,840 INFO:root:copying data_juicer/core/analyzer.py -> build/lib/data_juicer/core 2024-12-06T09:55:29,843 INFO:root:creating build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,844 INFO:root:copying data_juicer/ops/deduplicator/document_simhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,847 INFO:root:copying data_juicer/ops/deduplicator/ray_video_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,849 INFO:root:copying data_juicer/ops/deduplicator/ray_image_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,851 INFO:root:copying data_juicer/ops/deduplicator/ray_basic_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,853 INFO:root:copying data_juicer/ops/deduplicator/__init__.py -> build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,854 INFO:root:copying data_juicer/ops/deduplicator/document_minhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,857 INFO:root:copying data_juicer/ops/deduplicator/ray_document_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,859 INFO:root:copying data_juicer/ops/deduplicator/video_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,861 INFO:root:copying data_juicer/ops/deduplicator/document_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,863 INFO:root:copying data_juicer/ops/deduplicator/image_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-06T09:55:29,866 INFO:root:creating build/lib/data_juicer/ops/common 2024-12-06T09:55:29,867 INFO:root:copying data_juicer/ops/common/helper_func.py -> build/lib/data_juicer/ops/common 2024-12-06T09:55:29,869 INFO:root:copying data_juicer/ops/common/__init__.py -> build/lib/data_juicer/ops/common 2024-12-06T09:55:29,871 INFO:root:copying data_juicer/ops/common/special_characters.py -> build/lib/data_juicer/ops/common 2024-12-06T09:55:29,875 INFO:root:creating build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,876 INFO:root:copying data_juicer/ops/mapper/python_file_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,878 INFO:root:copying data_juicer/ops/mapper/remove_repeat_sentences_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,880 INFO:root:copying data_juicer/ops/mapper/python_lambda_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,882 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,885 INFO:root:copying data_juicer/ops/mapper/clean_copyright_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,887 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_video_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,889 INFO:root:copying data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,891 INFO:root:copying data_juicer/ops/mapper/chinese_convert_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,894 INFO:root:copying data_juicer/ops/mapper/remove_bibliography_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,896 INFO:root:copying data_juicer/ops/mapper/punctuation_normalization_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,898 INFO:root:copying data_juicer/ops/mapper/image_diffusion_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,900 INFO:root:copying data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,902 INFO:root:copying data_juicer/ops/mapper/remove_comments_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,904 INFO:root:copying data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,906 INFO:root:copying data_juicer/ops/mapper/remove_header_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,909 INFO:root:copying data_juicer/ops/mapper/clean_links_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,911 INFO:root:copying data_juicer/ops/mapper/optimize_query_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,913 INFO:root:copying data_juicer/ops/mapper/extract_entity_relation_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,915 INFO:root:copying data_juicer/ops/mapper/clean_html_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,918 INFO:root:copying data_juicer/ops/mapper/remove_table_text_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,920 INFO:root:copying data_juicer/ops/mapper/video_resize_resolution_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,922 INFO:root:copying data_juicer/ops/mapper/clean_ip_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,924 INFO:root:copying data_juicer/ops/mapper/optimize_response_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,926 INFO:root:copying data_juicer/ops/mapper/remove_non_chinese_character_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,927 INFO:root:copying data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,930 INFO:root:copying data_juicer/ops/mapper/whitespace_normalization_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,932 INFO:root:copying data_juicer/ops/mapper/clean_email_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,934 INFO:root:copying data_juicer/ops/mapper/image_face_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,936 INFO:root:copying data_juicer/ops/mapper/video_split_by_key_frame_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,938 INFO:root:copying data_juicer/ops/mapper/video_split_by_scene_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,941 INFO:root:copying data_juicer/ops/mapper/image_captioning_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,943 INFO:root:copying data_juicer/ops/mapper/extract_entity_attribute_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,945 INFO:root:copying data_juicer/ops/mapper/replace_content_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,947 INFO:root:copying data_juicer/ops/mapper/image_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,949 INFO:root:copying data_juicer/ops/mapper/__init__.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,951 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,954 INFO:root:copying data_juicer/ops/mapper/extract_event_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,956 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_audio_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,958 INFO:root:copying data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,961 INFO:root:copying data_juicer/ops/mapper/extract_nickname_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,963 INFO:root:copying data_juicer/ops/mapper/video_split_by_duration_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,966 INFO:root:copying data_juicer/ops/mapper/sentence_split_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,967 INFO:root:copying data_juicer/ops/mapper/calibrate_response_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,969 INFO:root:copying data_juicer/ops/mapper/calibrate_qa_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,972 INFO:root:copying data_juicer/ops/mapper/generate_qa_from_text_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,974 INFO:root:copying data_juicer/ops/mapper/nlpcda_zh_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,976 INFO:root:copying data_juicer/ops/mapper/fix_unicode_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,978 INFO:root:copying data_juicer/ops/mapper/extract_keyword_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,981 INFO:root:copying data_juicer/ops/mapper/generate_qa_from_examples_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,983 INFO:root:copying data_juicer/ops/mapper/remove_specific_chars_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,985 INFO:root:copying data_juicer/ops/mapper/calibrate_query_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,987 INFO:root:copying data_juicer/ops/mapper/optimize_qa_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,989 INFO:root:copying data_juicer/ops/mapper/pair_preference_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,991 INFO:root:copying data_juicer/ops/mapper/video_remove_watermark_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,994 INFO:root:copying data_juicer/ops/mapper/video_face_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,996 INFO:root:copying data_juicer/ops/mapper/remove_long_words_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:29,998 INFO:root:copying data_juicer/ops/mapper/image_tagging_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:30,000 INFO:root:copying data_juicer/ops/mapper/nlpaug_en_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:30,002 INFO:root:copying data_juicer/ops/mapper/video_tagging_from_audio_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:30,004 INFO:root:copying data_juicer/ops/mapper/expand_macro_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:30,006 INFO:root:copying data_juicer/ops/mapper/text_chunk_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:30,008 INFO:root:copying data_juicer/ops/mapper/video_tagging_from_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-06T09:55:30,012 INFO:root:creating build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,013 INFO:root:copying data_juicer/ops/filter/image_nsfw_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,015 INFO:root:copying data_juicer/ops/filter/perplexity_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,017 INFO:root:copying data_juicer/ops/filter/image_shape_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,019 INFO:root:copying data_juicer/ops/filter/image_size_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,021 INFO:root:copying data_juicer/ops/filter/audio_duration_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,022 INFO:root:copying data_juicer/ops/filter/phrase_grounding_recall_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,025 INFO:root:copying data_juicer/ops/filter/image_face_count_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,027 INFO:root:copying data_juicer/ops/filter/stopwords_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,030 INFO:root:copying data_juicer/ops/filter/specified_field_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,032 INFO:root:copying data_juicer/ops/filter/language_id_score_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,034 INFO:root:copying data_juicer/ops/filter/video_motion_score_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,036 INFO:root:copying data_juicer/ops/filter/token_num_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,038 INFO:root:copying data_juicer/ops/filter/character_repetition_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,040 INFO:root:copying data_juicer/ops/filter/average_line_length_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,042 INFO:root:copying data_juicer/ops/filter/text_action_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,044 INFO:root:copying data_juicer/ops/filter/image_pair_similarity_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,046 INFO:root:copying data_juicer/ops/filter/image_aspect_ratio_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,048 INFO:root:copying data_juicer/ops/filter/image_aesthetics_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,051 INFO:root:copying data_juicer/ops/filter/suffix_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,052 INFO:root:copying data_juicer/ops/filter/__init__.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,054 INFO:root:copying data_juicer/ops/filter/words_num_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,056 INFO:root:copying data_juicer/ops/filter/special_characters_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,058 INFO:root:copying data_juicer/ops/filter/text_entity_dependency_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,060 INFO:root:copying data_juicer/ops/filter/video_ocr_area_ratio_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,062 INFO:root:copying data_juicer/ops/filter/audio_nmf_snr_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,065 INFO:root:copying data_juicer/ops/filter/specified_numeric_field_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,066 INFO:root:copying data_juicer/ops/filter/video_duration_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,068 INFO:root:copying data_juicer/ops/filter/video_tagging_from_frames_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,070 INFO:root:copying data_juicer/ops/filter/image_watermark_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,072 INFO:root:copying data_juicer/ops/filter/image_text_similarity_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,074 INFO:root:copying data_juicer/ops/filter/video_aspect_ratio_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,076 INFO:root:copying data_juicer/ops/filter/video_aesthetics_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,078 INFO:root:copying data_juicer/ops/filter/word_repetition_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,081 INFO:root:copying data_juicer/ops/filter/audio_size_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,082 INFO:root:copying data_juicer/ops/filter/video_motion_score_raft_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,085 INFO:root:copying data_juicer/ops/filter/image_text_matching_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,087 INFO:root:copying data_juicer/ops/filter/maximum_line_length_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,089 INFO:root:copying data_juicer/ops/filter/alphanumeric_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,091 INFO:root:copying data_juicer/ops/filter/image_face_ratio_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,093 INFO:root:copying data_juicer/ops/filter/text_length_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,095 INFO:root:copying data_juicer/ops/filter/video_resolution_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,097 INFO:root:copying data_juicer/ops/filter/flagged_words_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,099 INFO:root:copying data_juicer/ops/filter/video_watermark_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,102 INFO:root:copying data_juicer/ops/filter/video_frames_text_similarity_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,104 INFO:root:copying data_juicer/ops/filter/video_nsfw_filter.py -> build/lib/data_juicer/ops/filter 2024-12-06T09:55:30,107 INFO:root:creating build/lib/data_juicer/ops/selector 2024-12-06T09:55:30,108 INFO:root:copying data_juicer/ops/selector/random_selector.py -> build/lib/data_juicer/ops/selector 2024-12-06T09:55:30,110 INFO:root:copying data_juicer/ops/selector/__init__.py -> build/lib/data_juicer/ops/selector 2024-12-06T09:55:30,112 INFO:root:copying data_juicer/ops/selector/frequency_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2024-12-06T09:55:30,114 INFO:root:copying data_juicer/ops/selector/range_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2024-12-06T09:55:30,117 INFO:root:copying data_juicer/ops/selector/topk_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2024-12-06T09:55:30,119 INFO:root:creating build/lib/data_juicer/tools 2024-12-06T09:55:30,120 INFO:root:copying tools/sandbox_starter.py -> build/lib/data_juicer/tools 2024-12-06T09:55:30,122 INFO:root:copying tools/__init__.py -> build/lib/data_juicer/tools 2024-12-06T09:55:30,124 INFO:root:copying tools/process_data.py -> build/lib/data_juicer/tools 2024-12-06T09:55:30,125 INFO:root:copying tools/analyze_data.py -> build/lib/data_juicer/tools 2024-12-06T09:55:30,174 /usr/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. 2024-12-06T09:55:30,175 !! 2024-12-06T09:55:30,176 ******************************************************************************** 2024-12-06T09:55:30,176 Please avoid running ``setup.py`` directly. 2024-12-06T09:55:30,177 Instead, use pypa/build, pypa/installer or other 2024-12-06T09:55:30,177 standards-based tools. 2024-12-06T09:55:30,178 See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. 2024-12-06T09:55:30,179 ******************************************************************************** 2024-12-06T09:55:30,180 !! 2024-12-06T09:55:30,180 self.initialize_options() 2024-12-06T09:55:30,202 INFO:root:installing to build/bdist.linux-armv7l/wheel 2024-12-06T09:55:30,203 INFO:root:running install 2024-12-06T09:55:30,228 INFO:root:running install_lib 2024-12-06T09:55:30,257 INFO:root:creating build/bdist.linux-armv7l/wheel 2024-12-06T09:55:30,259 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer 2024-12-06T09:55:30,261 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/format 2024-12-06T09:55:30,262 INFO:root:copying build/lib/data_juicer/format/load.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-06T09:55:30,264 INFO:root:copying build/lib/data_juicer/format/empty_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-06T09:55:30,266 INFO:root:copying build/lib/data_juicer/format/formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-06T09:55:30,269 INFO:root:copying build/lib/data_juicer/format/tsv_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-06T09:55:30,271 INFO:root:copying build/lib/data_juicer/format/csv_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-06T09:55:30,273 INFO:root:copying build/lib/data_juicer/format/mixture_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-06T09:55:30,275 INFO:root:copying build/lib/data_juicer/format/parquet_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-06T09:55:30,277 INFO:root:copying build/lib/data_juicer/format/json_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-06T09:55:30,279 INFO:root:copying build/lib/data_juicer/format/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-06T09:55:30,281 INFO:root:copying build/lib/data_juicer/format/text_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-06T09:55:30,284 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/analysis 2024-12-06T09:55:30,285 INFO:root:copying build/lib/data_juicer/analysis/collector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-06T09:55:30,287 INFO:root:copying build/lib/data_juicer/analysis/draw.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-06T09:55:30,289 INFO:root:copying build/lib/data_juicer/analysis/diversity_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-06T09:55:30,291 INFO:root:copying build/lib/data_juicer/analysis/overall_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-06T09:55:30,294 INFO:root:copying build/lib/data_juicer/analysis/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-06T09:55:30,296 INFO:root:copying build/lib/data_juicer/analysis/measure.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-06T09:55:30,298 INFO:root:copying build/lib/data_juicer/analysis/column_wise_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-06T09:55:30,301 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/utils 2024-12-06T09:55:30,302 INFO:root:copying build/lib/data_juicer/utils/registry.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,304 INFO:root:copying build/lib/data_juicer/utils/availability_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,306 INFO:root:copying build/lib/data_juicer/utils/logger_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,308 INFO:root:copying build/lib/data_juicer/utils/resource_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,310 INFO:root:copying build/lib/data_juicer/utils/constant.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,314 INFO:root:copying build/lib/data_juicer/utils/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,318 INFO:root:copying build/lib/data_juicer/utils/ckpt_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,321 INFO:root:copying build/lib/data_juicer/utils/file_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,323 INFO:root:copying build/lib/data_juicer/utils/unittest_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,325 INFO:root:copying build/lib/data_juicer/utils/asset_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,327 INFO:root:copying build/lib/data_juicer/utils/model_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,331 INFO:root:copying build/lib/data_juicer/utils/auto_install_mapping.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,334 INFO:root:copying build/lib/data_juicer/utils/mm_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,336 INFO:root:copying build/lib/data_juicer/utils/fingerprint_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,339 INFO:root:copying build/lib/data_juicer/utils/auto_install_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,341 INFO:root:copying build/lib/data_juicer/utils/compress.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,343 INFO:root:copying build/lib/data_juicer/utils/process_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,345 INFO:root:copying build/lib/data_juicer/utils/common_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,347 INFO:root:copying build/lib/data_juicer/utils/cache_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,351 INFO:root:copying build/lib/data_juicer/utils/lazy_loader.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-06T09:55:30,354 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/config 2024-12-06T09:55:30,356 INFO:root:copying build/lib/data_juicer/config/config.py -> build/bdist.linux-armv7l/wheel/./data_juicer/config 2024-12-06T09:55:30,359 INFO:root:copying build/lib/data_juicer/config/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/config 2024-12-06T09:55:30,362 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops 2024-12-06T09:55:30,363 INFO:root:copying build/lib/data_juicer/ops/load.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2024-12-06T09:55:30,366 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/deduplicator 2024-12-06T09:55:30,367 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_simhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-06T09:55:30,371 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_video_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-06T09:55:30,373 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_image_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-06T09:55:30,376 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_basic_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-06T09:55:30,378 INFO:root:copying build/lib/data_juicer/ops/deduplicator/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-06T09:55:30,380 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_minhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-06T09:55:30,382 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_document_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-06T09:55:30,384 INFO:root:copying build/lib/data_juicer/ops/deduplicator/video_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-06T09:55:30,386 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-06T09:55:30,389 INFO:root:copying build/lib/data_juicer/ops/deduplicator/image_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-06T09:55:30,391 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/common 2024-12-06T09:55:30,393 INFO:root:copying build/lib/data_juicer/ops/common/helper_func.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2024-12-06T09:55:30,395 INFO:root:copying build/lib/data_juicer/ops/common/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2024-12-06T09:55:30,397 INFO:root:copying build/lib/data_juicer/ops/common/special_characters.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2024-12-06T09:55:30,401 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/mapper 2024-12-06T09:55:30,402 INFO:root:copying build/lib/data_juicer/ops/mapper/python_file_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,405 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_repeat_sentences_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,407 INFO:root:copying build/lib/data_juicer/ops/mapper/python_lambda_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,409 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,411 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_copyright_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,413 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_video_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,416 INFO:root:copying build/lib/data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,418 INFO:root:copying build/lib/data_juicer/ops/mapper/chinese_convert_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,420 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_bibliography_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,422 INFO:root:copying build/lib/data_juicer/ops/mapper/punctuation_normalization_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,424 INFO:root:copying build/lib/data_juicer/ops/mapper/image_diffusion_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,426 INFO:root:copying build/lib/data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,428 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_comments_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,430 INFO:root:copying build/lib/data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,432 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_header_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,434 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_links_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,436 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_query_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,437 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_entity_relation_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,440 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_html_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,442 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_table_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,444 INFO:root:copying build/lib/data_juicer/ops/mapper/video_resize_resolution_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,446 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_ip_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,448 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_response_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,450 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_non_chinese_character_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,452 INFO:root:copying build/lib/data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,454 INFO:root:copying build/lib/data_juicer/ops/mapper/whitespace_normalization_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,456 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_email_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,458 INFO:root:copying build/lib/data_juicer/ops/mapper/image_face_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,460 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_key_frame_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,462 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_scene_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,465 INFO:root:copying build/lib/data_juicer/ops/mapper/image_captioning_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,467 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_entity_attribute_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,470 INFO:root:copying build/lib/data_juicer/ops/mapper/replace_content_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,471 INFO:root:copying build/lib/data_juicer/ops/mapper/image_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,473 INFO:root:copying build/lib/data_juicer/ops/mapper/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,475 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,478 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_event_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,481 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_audio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,483 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,485 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_nickname_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,487 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_duration_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,489 INFO:root:copying build/lib/data_juicer/ops/mapper/sentence_split_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,491 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_response_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,493 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_qa_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,495 INFO:root:copying build/lib/data_juicer/ops/mapper/generate_qa_from_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,497 INFO:root:copying build/lib/data_juicer/ops/mapper/nlpcda_zh_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,499 INFO:root:copying build/lib/data_juicer/ops/mapper/fix_unicode_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,501 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_keyword_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,504 INFO:root:copying build/lib/data_juicer/ops/mapper/generate_qa_from_examples_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,506 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_specific_chars_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,508 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_query_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,510 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_qa_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,512 INFO:root:copying build/lib/data_juicer/ops/mapper/pair_preference_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,514 INFO:root:copying build/lib/data_juicer/ops/mapper/video_remove_watermark_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,517 INFO:root:copying build/lib/data_juicer/ops/mapper/video_face_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,519 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_long_words_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,521 INFO:root:copying build/lib/data_juicer/ops/mapper/image_tagging_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,523 INFO:root:copying build/lib/data_juicer/ops/mapper/nlpaug_en_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,525 INFO:root:copying build/lib/data_juicer/ops/mapper/video_tagging_from_audio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,527 INFO:root:copying build/lib/data_juicer/ops/mapper/expand_macro_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,529 INFO:root:copying build/lib/data_juicer/ops/mapper/text_chunk_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,531 INFO:root:copying build/lib/data_juicer/ops/mapper/video_tagging_from_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-06T09:55:30,535 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/filter 2024-12-06T09:55:30,536 INFO:root:copying build/lib/data_juicer/ops/filter/image_nsfw_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,538 INFO:root:copying build/lib/data_juicer/ops/filter/perplexity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,540 INFO:root:copying build/lib/data_juicer/ops/filter/image_shape_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,542 INFO:root:copying build/lib/data_juicer/ops/filter/image_size_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,544 INFO:root:copying build/lib/data_juicer/ops/filter/audio_duration_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,546 INFO:root:copying build/lib/data_juicer/ops/filter/phrase_grounding_recall_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,548 INFO:root:copying build/lib/data_juicer/ops/filter/image_face_count_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,550 INFO:root:copying build/lib/data_juicer/ops/filter/stopwords_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,552 INFO:root:copying build/lib/data_juicer/ops/filter/specified_field_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,554 INFO:root:copying build/lib/data_juicer/ops/filter/language_id_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,556 INFO:root:copying build/lib/data_juicer/ops/filter/video_motion_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,558 INFO:root:copying build/lib/data_juicer/ops/filter/token_num_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,560 INFO:root:copying build/lib/data_juicer/ops/filter/character_repetition_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,562 INFO:root:copying build/lib/data_juicer/ops/filter/average_line_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,564 INFO:root:copying build/lib/data_juicer/ops/filter/text_action_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,566 INFO:root:copying build/lib/data_juicer/ops/filter/image_pair_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,568 INFO:root:copying build/lib/data_juicer/ops/filter/image_aspect_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,570 INFO:root:copying build/lib/data_juicer/ops/filter/image_aesthetics_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,572 INFO:root:copying build/lib/data_juicer/ops/filter/suffix_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,574 INFO:root:copying build/lib/data_juicer/ops/filter/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,575 INFO:root:copying build/lib/data_juicer/ops/filter/words_num_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,577 INFO:root:copying build/lib/data_juicer/ops/filter/special_characters_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,579 INFO:root:copying build/lib/data_juicer/ops/filter/text_entity_dependency_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,581 INFO:root:copying build/lib/data_juicer/ops/filter/video_ocr_area_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,583 INFO:root:copying build/lib/data_juicer/ops/filter/audio_nmf_snr_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,586 INFO:root:copying build/lib/data_juicer/ops/filter/specified_numeric_field_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,588 INFO:root:copying build/lib/data_juicer/ops/filter/video_duration_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,590 INFO:root:copying build/lib/data_juicer/ops/filter/video_tagging_from_frames_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,592 INFO:root:copying build/lib/data_juicer/ops/filter/image_watermark_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,594 INFO:root:copying build/lib/data_juicer/ops/filter/image_text_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,597 INFO:root:copying build/lib/data_juicer/ops/filter/video_aspect_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,599 INFO:root:copying build/lib/data_juicer/ops/filter/video_aesthetics_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,601 INFO:root:copying build/lib/data_juicer/ops/filter/word_repetition_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,603 INFO:root:copying build/lib/data_juicer/ops/filter/audio_size_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,605 INFO:root:copying build/lib/data_juicer/ops/filter/video_motion_score_raft_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,608 INFO:root:copying build/lib/data_juicer/ops/filter/image_text_matching_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,611 INFO:root:copying build/lib/data_juicer/ops/filter/maximum_line_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,613 INFO:root:copying build/lib/data_juicer/ops/filter/alphanumeric_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,615 INFO:root:copying build/lib/data_juicer/ops/filter/image_face_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,617 INFO:root:copying build/lib/data_juicer/ops/filter/text_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,619 INFO:root:copying build/lib/data_juicer/ops/filter/video_resolution_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,621 INFO:root:copying build/lib/data_juicer/ops/filter/flagged_words_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,623 INFO:root:copying build/lib/data_juicer/ops/filter/video_watermark_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,625 INFO:root:copying build/lib/data_juicer/ops/filter/video_frames_text_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,628 INFO:root:copying build/lib/data_juicer/ops/filter/video_nsfw_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-06T09:55:30,630 INFO:root:copying build/lib/data_juicer/ops/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2024-12-06T09:55:30,631 INFO:root:copying build/lib/data_juicer/ops/op_fusion.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2024-12-06T09:55:30,633 INFO:root:copying build/lib/data_juicer/ops/base_op.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2024-12-06T09:55:30,637 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/selector 2024-12-06T09:55:30,638 INFO:root:copying build/lib/data_juicer/ops/selector/random_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2024-12-06T09:55:30,640 INFO:root:copying build/lib/data_juicer/ops/selector/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2024-12-06T09:55:30,641 INFO:root:copying build/lib/data_juicer/ops/selector/frequency_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2024-12-06T09:55:30,643 INFO:root:copying build/lib/data_juicer/ops/selector/range_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2024-12-06T09:55:30,646 INFO:root:copying build/lib/data_juicer/ops/selector/topk_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2024-12-06T09:55:30,647 INFO:root:copying build/lib/data_juicer/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer 2024-12-06T09:55:30,650 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/core 2024-12-06T09:55:30,651 INFO:root:copying build/lib/data_juicer/core/executor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-06T09:55:30,654 INFO:root:copying build/lib/data_juicer/core/tracer.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-06T09:55:30,656 INFO:root:copying build/lib/data_juicer/core/data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-06T09:55:30,658 INFO:root:copying build/lib/data_juicer/core/exporter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-06T09:55:30,660 INFO:root:copying build/lib/data_juicer/core/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-06T09:55:30,662 INFO:root:copying build/lib/data_juicer/core/ray_executor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-06T09:55:30,664 INFO:root:copying build/lib/data_juicer/core/monitor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-06T09:55:30,666 INFO:root:copying build/lib/data_juicer/core/adapter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-06T09:55:30,668 INFO:root:copying build/lib/data_juicer/core/ray_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-06T09:55:30,670 INFO:root:copying build/lib/data_juicer/core/analyzer.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-06T09:55:30,673 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/tools 2024-12-06T09:55:30,674 INFO:root:copying build/lib/data_juicer/tools/sandbox_starter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2024-12-06T09:55:30,676 INFO:root:copying build/lib/data_juicer/tools/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2024-12-06T09:55:30,678 INFO:root:copying build/lib/data_juicer/tools/process_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2024-12-06T09:55:30,679 INFO:root:copying build/lib/data_juicer/tools/analyze_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2024-12-06T09:55:30,681 INFO:root:running install_egg_info 2024-12-06T09:55:30,714 INFO:root:running egg_info 2024-12-06T09:55:30,743 INFO:root:writing py_data_juicer.egg-info/PKG-INFO 2024-12-06T09:55:30,747 INFO:root:writing dependency_links to py_data_juicer.egg-info/dependency_links.txt 2024-12-06T09:55:30,749 INFO:root:writing entry points to py_data_juicer.egg-info/entry_points.txt 2024-12-06T09:55:30,751 INFO:root:writing requirements to py_data_juicer.egg-info/requires.txt 2024-12-06T09:55:30,752 INFO:root:writing top-level names to py_data_juicer.egg-info/top_level.txt 2024-12-06T09:55:30,803 INFO:root:reading manifest file 'py_data_juicer.egg-info/SOURCES.txt' 2024-12-06T09:55:30,814 INFO:root:adding license file 'LICENSE' 2024-12-06T09:55:30,824 INFO:root:writing manifest file 'py_data_juicer.egg-info/SOURCES.txt' 2024-12-06T09:55:30,826 INFO:root:Copying py_data_juicer.egg-info to build/bdist.linux-armv7l/wheel/./py_data_juicer-1.0.1-py3.11.egg-info 2024-12-06T09:55:30,839 INFO:root:running install_scripts 2024-12-06T09:55:30,858 INFO:root:creating build/bdist.linux-armv7l/wheel/py_data_juicer-1.0.1.dist-info/WHEEL 2024-12-06T09:55:30,861 INFO:wheel:creating '/tmp/pip-wheel-mmgjowf0/py_data_juicer-1.0.1-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2024-12-06T09:55:30,863 INFO:wheel:adding 'data_juicer/__init__.py' 2024-12-06T09:55:30,865 INFO:wheel:adding 'data_juicer/analysis/__init__.py' 2024-12-06T09:55:30,866 INFO:wheel:adding 'data_juicer/analysis/collector.py' 2024-12-06T09:55:30,868 INFO:wheel:adding 'data_juicer/analysis/column_wise_analysis.py' 2024-12-06T09:55:30,870 INFO:wheel:adding 'data_juicer/analysis/diversity_analysis.py' 2024-12-06T09:55:30,871 INFO:wheel:adding 'data_juicer/analysis/draw.py' 2024-12-06T09:55:30,872 INFO:wheel:adding 'data_juicer/analysis/measure.py' 2024-12-06T09:55:30,874 INFO:wheel:adding 'data_juicer/analysis/overall_analysis.py' 2024-12-06T09:55:30,875 INFO:wheel:adding 'data_juicer/config/__init__.py' 2024-12-06T09:55:30,879 INFO:wheel:adding 'data_juicer/config/config.py' 2024-12-06T09:55:30,881 INFO:wheel:adding 'data_juicer/core/__init__.py' 2024-12-06T09:55:30,883 INFO:wheel:adding 'data_juicer/core/adapter.py' 2024-12-06T09:55:30,885 INFO:wheel:adding 'data_juicer/core/analyzer.py' 2024-12-06T09:55:30,887 INFO:wheel:adding 'data_juicer/core/data.py' 2024-12-06T09:55:30,889 INFO:wheel:adding 'data_juicer/core/executor.py' 2024-12-06T09:55:30,891 INFO:wheel:adding 'data_juicer/core/exporter.py' 2024-12-06T09:55:30,892 INFO:wheel:adding 'data_juicer/core/monitor.py' 2024-12-06T09:55:30,894 INFO:wheel:adding 'data_juicer/core/ray_data.py' 2024-12-06T09:55:30,895 INFO:wheel:adding 'data_juicer/core/ray_executor.py' 2024-12-06T09:55:30,897 INFO:wheel:adding 'data_juicer/core/tracer.py' 2024-12-06T09:55:30,899 INFO:wheel:adding 'data_juicer/format/__init__.py' 2024-12-06T09:55:30,900 INFO:wheel:adding 'data_juicer/format/csv_formatter.py' 2024-12-06T09:55:30,901 INFO:wheel:adding 'data_juicer/format/empty_formatter.py' 2024-12-06T09:55:30,903 INFO:wheel:adding 'data_juicer/format/formatter.py' 2024-12-06T09:55:30,905 INFO:wheel:adding 'data_juicer/format/json_formatter.py' 2024-12-06T09:55:30,906 INFO:wheel:adding 'data_juicer/format/load.py' 2024-12-06T09:55:30,907 INFO:wheel:adding 'data_juicer/format/mixture_formatter.py' 2024-12-06T09:55:30,909 INFO:wheel:adding 'data_juicer/format/parquet_formatter.py' 2024-12-06T09:55:30,910 INFO:wheel:adding 'data_juicer/format/text_formatter.py' 2024-12-06T09:55:30,911 INFO:wheel:adding 'data_juicer/format/tsv_formatter.py' 2024-12-06T09:55:30,913 INFO:wheel:adding 'data_juicer/ops/__init__.py' 2024-12-06T09:55:30,915 INFO:wheel:adding 'data_juicer/ops/base_op.py' 2024-12-06T09:55:30,917 INFO:wheel:adding 'data_juicer/ops/load.py' 2024-12-06T09:55:30,918 INFO:wheel:adding 'data_juicer/ops/op_fusion.py' 2024-12-06T09:55:30,920 INFO:wheel:adding 'data_juicer/ops/common/__init__.py' 2024-12-06T09:55:30,922 INFO:wheel:adding 'data_juicer/ops/common/helper_func.py' 2024-12-06T09:55:30,923 INFO:wheel:adding 'data_juicer/ops/common/special_characters.py' 2024-12-06T09:55:30,925 INFO:wheel:adding 'data_juicer/ops/deduplicator/__init__.py' 2024-12-06T09:55:30,926 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_deduplicator.py' 2024-12-06T09:55:30,928 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_minhash_deduplicator.py' 2024-12-06T09:55:30,930 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_simhash_deduplicator.py' 2024-12-06T09:55:30,932 INFO:wheel:adding 'data_juicer/ops/deduplicator/image_deduplicator.py' 2024-12-06T09:55:30,933 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_basic_deduplicator.py' 2024-12-06T09:55:30,935 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_document_deduplicator.py' 2024-12-06T09:55:30,936 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_image_deduplicator.py' 2024-12-06T09:55:30,937 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_video_deduplicator.py' 2024-12-06T09:55:30,939 INFO:wheel:adding 'data_juicer/ops/deduplicator/video_deduplicator.py' 2024-12-06T09:55:30,941 INFO:wheel:adding 'data_juicer/ops/filter/__init__.py' 2024-12-06T09:55:30,943 INFO:wheel:adding 'data_juicer/ops/filter/alphanumeric_filter.py' 2024-12-06T09:55:30,944 INFO:wheel:adding 'data_juicer/ops/filter/audio_duration_filter.py' 2024-12-06T09:55:30,946 INFO:wheel:adding 'data_juicer/ops/filter/audio_nmf_snr_filter.py' 2024-12-06T09:55:30,947 INFO:wheel:adding 'data_juicer/ops/filter/audio_size_filter.py' 2024-12-06T09:55:30,948 INFO:wheel:adding 'data_juicer/ops/filter/average_line_length_filter.py' 2024-12-06T09:55:30,950 INFO:wheel:adding 'data_juicer/ops/filter/character_repetition_filter.py' 2024-12-06T09:55:30,951 INFO:wheel:adding 'data_juicer/ops/filter/flagged_words_filter.py' 2024-12-06T09:55:30,953 INFO:wheel:adding 'data_juicer/ops/filter/image_aesthetics_filter.py' 2024-12-06T09:55:30,954 INFO:wheel:adding 'data_juicer/ops/filter/image_aspect_ratio_filter.py' 2024-12-06T09:55:30,956 INFO:wheel:adding 'data_juicer/ops/filter/image_face_count_filter.py' 2024-12-06T09:55:30,957 INFO:wheel:adding 'data_juicer/ops/filter/image_face_ratio_filter.py' 2024-12-06T09:55:30,959 INFO:wheel:adding 'data_juicer/ops/filter/image_nsfw_filter.py' 2024-12-06T09:55:30,960 INFO:wheel:adding 'data_juicer/ops/filter/image_pair_similarity_filter.py' 2024-12-06T09:55:30,962 INFO:wheel:adding 'data_juicer/ops/filter/image_shape_filter.py' 2024-12-06T09:55:30,963 INFO:wheel:adding 'data_juicer/ops/filter/image_size_filter.py' 2024-12-06T09:55:30,965 INFO:wheel:adding 'data_juicer/ops/filter/image_text_matching_filter.py' 2024-12-06T09:55:30,966 INFO:wheel:adding 'data_juicer/ops/filter/image_text_similarity_filter.py' 2024-12-06T09:55:30,968 INFO:wheel:adding 'data_juicer/ops/filter/image_watermark_filter.py' 2024-12-06T09:55:30,969 INFO:wheel:adding 'data_juicer/ops/filter/language_id_score_filter.py' 2024-12-06T09:55:30,970 INFO:wheel:adding 'data_juicer/ops/filter/maximum_line_length_filter.py' 2024-12-06T09:55:30,972 INFO:wheel:adding 'data_juicer/ops/filter/perplexity_filter.py' 2024-12-06T09:55:30,974 INFO:wheel:adding 'data_juicer/ops/filter/phrase_grounding_recall_filter.py' 2024-12-06T09:55:30,975 INFO:wheel:adding 'data_juicer/ops/filter/special_characters_filter.py' 2024-12-06T09:55:30,977 INFO:wheel:adding 'data_juicer/ops/filter/specified_field_filter.py' 2024-12-06T09:55:30,978 INFO:wheel:adding 'data_juicer/ops/filter/specified_numeric_field_filter.py' 2024-12-06T09:55:30,980 INFO:wheel:adding 'data_juicer/ops/filter/stopwords_filter.py' 2024-12-06T09:55:30,981 INFO:wheel:adding 'data_juicer/ops/filter/suffix_filter.py' 2024-12-06T09:55:30,983 INFO:wheel:adding 'data_juicer/ops/filter/text_action_filter.py' 2024-12-06T09:55:30,984 INFO:wheel:adding 'data_juicer/ops/filter/text_entity_dependency_filter.py' 2024-12-06T09:55:30,985 INFO:wheel:adding 'data_juicer/ops/filter/text_length_filter.py' 2024-12-06T09:55:30,987 INFO:wheel:adding 'data_juicer/ops/filter/token_num_filter.py' 2024-12-06T09:55:30,989 INFO:wheel:adding 'data_juicer/ops/filter/video_aesthetics_filter.py' 2024-12-06T09:55:30,990 INFO:wheel:adding 'data_juicer/ops/filter/video_aspect_ratio_filter.py' 2024-12-06T09:55:30,991 INFO:wheel:adding 'data_juicer/ops/filter/video_duration_filter.py' 2024-12-06T09:55:30,993 INFO:wheel:adding 'data_juicer/ops/filter/video_frames_text_similarity_filter.py' 2024-12-06T09:55:30,995 INFO:wheel:adding 'data_juicer/ops/filter/video_motion_score_filter.py' 2024-12-06T09:55:30,996 INFO:wheel:adding 'data_juicer/ops/filter/video_motion_score_raft_filter.py' 2024-12-06T09:55:30,998 INFO:wheel:adding 'data_juicer/ops/filter/video_nsfw_filter.py' 2024-12-06T09:55:30,999 INFO:wheel:adding 'data_juicer/ops/filter/video_ocr_area_ratio_filter.py' 2024-12-06T09:55:31,001 INFO:wheel:adding 'data_juicer/ops/filter/video_resolution_filter.py' 2024-12-06T09:55:31,002 INFO:wheel:adding 'data_juicer/ops/filter/video_tagging_from_frames_filter.py' 2024-12-06T09:55:31,004 INFO:wheel:adding 'data_juicer/ops/filter/video_watermark_filter.py' 2024-12-06T09:55:31,006 INFO:wheel:adding 'data_juicer/ops/filter/word_repetition_filter.py' 2024-12-06T09:55:31,007 INFO:wheel:adding 'data_juicer/ops/filter/words_num_filter.py' 2024-12-06T09:55:31,010 INFO:wheel:adding 'data_juicer/ops/mapper/__init__.py' 2024-12-06T09:55:31,011 INFO:wheel:adding 'data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py' 2024-12-06T09:55:31,013 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_qa_mapper.py' 2024-12-06T09:55:31,014 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_query_mapper.py' 2024-12-06T09:55:31,015 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_response_mapper.py' 2024-12-06T09:55:31,017 INFO:wheel:adding 'data_juicer/ops/mapper/chinese_convert_mapper.py' 2024-12-06T09:55:31,018 INFO:wheel:adding 'data_juicer/ops/mapper/clean_copyright_mapper.py' 2024-12-06T09:55:31,019 INFO:wheel:adding 'data_juicer/ops/mapper/clean_email_mapper.py' 2024-12-06T09:55:31,020 INFO:wheel:adding 'data_juicer/ops/mapper/clean_html_mapper.py' 2024-12-06T09:55:31,022 INFO:wheel:adding 'data_juicer/ops/mapper/clean_ip_mapper.py' 2024-12-06T09:55:31,023 INFO:wheel:adding 'data_juicer/ops/mapper/clean_links_mapper.py' 2024-12-06T09:55:31,024 INFO:wheel:adding 'data_juicer/ops/mapper/expand_macro_mapper.py' 2024-12-06T09:55:31,026 INFO:wheel:adding 'data_juicer/ops/mapper/extract_entity_attribute_mapper.py' 2024-12-06T09:55:31,029 INFO:wheel:adding 'data_juicer/ops/mapper/extract_entity_relation_mapper.py' 2024-12-06T09:55:31,031 INFO:wheel:adding 'data_juicer/ops/mapper/extract_event_mapper.py' 2024-12-06T09:55:31,033 INFO:wheel:adding 'data_juicer/ops/mapper/extract_keyword_mapper.py' 2024-12-06T09:55:31,034 INFO:wheel:adding 'data_juicer/ops/mapper/extract_nickname_mapper.py' 2024-12-06T09:55:31,036 INFO:wheel:adding 'data_juicer/ops/mapper/fix_unicode_mapper.py' 2024-12-06T09:55:31,038 INFO:wheel:adding 'data_juicer/ops/mapper/generate_qa_from_examples_mapper.py' 2024-12-06T09:55:31,039 INFO:wheel:adding 'data_juicer/ops/mapper/generate_qa_from_text_mapper.py' 2024-12-06T09:55:31,041 INFO:wheel:adding 'data_juicer/ops/mapper/image_blur_mapper.py' 2024-12-06T09:55:31,043 INFO:wheel:adding 'data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py' 2024-12-06T09:55:31,046 INFO:wheel:adding 'data_juicer/ops/mapper/image_captioning_mapper.py' 2024-12-06T09:55:31,048 INFO:wheel:adding 'data_juicer/ops/mapper/image_diffusion_mapper.py' 2024-12-06T09:55:31,049 INFO:wheel:adding 'data_juicer/ops/mapper/image_face_blur_mapper.py' 2024-12-06T09:55:31,051 INFO:wheel:adding 'data_juicer/ops/mapper/image_tagging_mapper.py' 2024-12-06T09:55:31,052 INFO:wheel:adding 'data_juicer/ops/mapper/nlpaug_en_mapper.py' 2024-12-06T09:55:31,054 INFO:wheel:adding 'data_juicer/ops/mapper/nlpcda_zh_mapper.py' 2024-12-06T09:55:31,055 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_qa_mapper.py' 2024-12-06T09:55:31,057 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_query_mapper.py' 2024-12-06T09:55:31,058 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_response_mapper.py' 2024-12-06T09:55:31,059 INFO:wheel:adding 'data_juicer/ops/mapper/pair_preference_mapper.py' 2024-12-06T09:55:31,061 INFO:wheel:adding 'data_juicer/ops/mapper/punctuation_normalization_mapper.py' 2024-12-06T09:55:31,062 INFO:wheel:adding 'data_juicer/ops/mapper/python_file_mapper.py' 2024-12-06T09:55:31,063 INFO:wheel:adding 'data_juicer/ops/mapper/python_lambda_mapper.py' 2024-12-06T09:55:31,064 INFO:wheel:adding 'data_juicer/ops/mapper/remove_bibliography_mapper.py' 2024-12-06T09:55:31,066 INFO:wheel:adding 'data_juicer/ops/mapper/remove_comments_mapper.py' 2024-12-06T09:55:31,067 INFO:wheel:adding 'data_juicer/ops/mapper/remove_header_mapper.py' 2024-12-06T09:55:31,068 INFO:wheel:adding 'data_juicer/ops/mapper/remove_long_words_mapper.py' 2024-12-06T09:55:31,069 INFO:wheel:adding 'data_juicer/ops/mapper/remove_non_chinese_character_mapper.py' 2024-12-06T09:55:31,071 INFO:wheel:adding 'data_juicer/ops/mapper/remove_repeat_sentences_mapper.py' 2024-12-06T09:55:31,072 INFO:wheel:adding 'data_juicer/ops/mapper/remove_specific_chars_mapper.py' 2024-12-06T09:55:31,073 INFO:wheel:adding 'data_juicer/ops/mapper/remove_table_text_mapper.py' 2024-12-06T09:55:31,075 INFO:wheel:adding 'data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py' 2024-12-06T09:55:31,076 INFO:wheel:adding 'data_juicer/ops/mapper/replace_content_mapper.py' 2024-12-06T09:55:31,077 INFO:wheel:adding 'data_juicer/ops/mapper/sentence_split_mapper.py' 2024-12-06T09:55:31,079 INFO:wheel:adding 'data_juicer/ops/mapper/text_chunk_mapper.py' 2024-12-06T09:55:31,080 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_audio_mapper.py' 2024-12-06T09:55:31,082 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_frames_mapper.py' 2024-12-06T09:55:31,084 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py' 2024-12-06T09:55:31,087 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_video_mapper.py' 2024-12-06T09:55:31,088 INFO:wheel:adding 'data_juicer/ops/mapper/video_face_blur_mapper.py' 2024-12-06T09:55:31,089 INFO:wheel:adding 'data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py' 2024-12-06T09:55:31,091 INFO:wheel:adding 'data_juicer/ops/mapper/video_remove_watermark_mapper.py' 2024-12-06T09:55:31,093 INFO:wheel:adding 'data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py' 2024-12-06T09:55:31,094 INFO:wheel:adding 'data_juicer/ops/mapper/video_resize_resolution_mapper.py' 2024-12-06T09:55:31,096 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_duration_mapper.py' 2024-12-06T09:55:31,098 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_key_frame_mapper.py' 2024-12-06T09:55:31,099 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_scene_mapper.py' 2024-12-06T09:55:31,100 INFO:wheel:adding 'data_juicer/ops/mapper/video_tagging_from_audio_mapper.py' 2024-12-06T09:55:31,102 INFO:wheel:adding 'data_juicer/ops/mapper/video_tagging_from_frames_mapper.py' 2024-12-06T09:55:31,103 INFO:wheel:adding 'data_juicer/ops/mapper/whitespace_normalization_mapper.py' 2024-12-06T09:55:31,105 INFO:wheel:adding 'data_juicer/ops/selector/__init__.py' 2024-12-06T09:55:31,106 INFO:wheel:adding 'data_juicer/ops/selector/frequency_specified_field_selector.py' 2024-12-06T09:55:31,108 INFO:wheel:adding 'data_juicer/ops/selector/random_selector.py' 2024-12-06T09:55:31,109 INFO:wheel:adding 'data_juicer/ops/selector/range_specified_field_selector.py' 2024-12-06T09:55:31,111 INFO:wheel:adding 'data_juicer/ops/selector/topk_specified_field_selector.py' 2024-12-06T09:55:31,112 INFO:wheel:adding 'data_juicer/tools/__init__.py' 2024-12-06T09:55:31,113 INFO:wheel:adding 'data_juicer/tools/analyze_data.py' 2024-12-06T09:55:31,114 INFO:wheel:adding 'data_juicer/tools/process_data.py' 2024-12-06T09:55:31,116 INFO:wheel:adding 'data_juicer/tools/sandbox_starter.py' 2024-12-06T09:55:31,118 INFO:wheel:adding 'data_juicer/utils/__init__.py' 2024-12-06T09:55:31,119 INFO:wheel:adding 'data_juicer/utils/asset_utils.py' 2024-12-06T09:55:31,120 INFO:wheel:adding 'data_juicer/utils/auto_install_mapping.py' 2024-12-06T09:55:31,122 INFO:wheel:adding 'data_juicer/utils/auto_install_utils.py' 2024-12-06T09:55:31,123 INFO:wheel:adding 'data_juicer/utils/availability_utils.py' 2024-12-06T09:55:31,124 INFO:wheel:adding 'data_juicer/utils/cache_utils.py' 2024-12-06T09:55:31,126 INFO:wheel:adding 'data_juicer/utils/ckpt_utils.py' 2024-12-06T09:55:31,127 INFO:wheel:adding 'data_juicer/utils/common_utils.py' 2024-12-06T09:55:31,129 INFO:wheel:adding 'data_juicer/utils/compress.py' 2024-12-06T09:55:31,131 INFO:wheel:adding 'data_juicer/utils/constant.py' 2024-12-06T09:55:31,133 INFO:wheel:adding 'data_juicer/utils/file_utils.py' 2024-12-06T09:55:31,134 INFO:wheel:adding 'data_juicer/utils/fingerprint_utils.py' 2024-12-06T09:55:31,136 INFO:wheel:adding 'data_juicer/utils/lazy_loader.py' 2024-12-06T09:55:31,137 INFO:wheel:adding 'data_juicer/utils/logger_utils.py' 2024-12-06T09:55:31,141 INFO:wheel:adding 'data_juicer/utils/mm_utils.py' 2024-12-06T09:55:31,145 INFO:wheel:adding 'data_juicer/utils/model_utils.py' 2024-12-06T09:55:31,147 INFO:wheel:adding 'data_juicer/utils/process_utils.py' 2024-12-06T09:55:31,148 INFO:wheel:adding 'data_juicer/utils/registry.py' 2024-12-06T09:55:31,149 INFO:wheel:adding 'data_juicer/utils/resource_utils.py' 2024-12-06T09:55:31,151 INFO:wheel:adding 'data_juicer/utils/unittest_utils.py' 2024-12-06T09:55:31,154 INFO:wheel:adding 'py_data_juicer-1.0.1.dist-info/LICENSE' 2024-12-06T09:55:31,158 INFO:wheel:adding 'py_data_juicer-1.0.1.dist-info/METADATA' 2024-12-06T09:55:31,159 INFO:wheel:adding 'py_data_juicer-1.0.1.dist-info/WHEEL' 2024-12-06T09:55:31,160 INFO:wheel:adding 'py_data_juicer-1.0.1.dist-info/entry_points.txt' 2024-12-06T09:55:31,161 INFO:wheel:adding 'py_data_juicer-1.0.1.dist-info/top_level.txt' 2024-12-06T09:55:31,164 INFO:wheel:adding 'py_data_juicer-1.0.1.dist-info/RECORD' 2024-12-06T09:55:31,171 INFO:root:removing build/bdist.linux-armv7l/wheel 2024-12-06T09:55:31,306 Building wheel for py-data-juicer (setup.py): finished with status 'done' 2024-12-06T09:55:31,312 Created wheel for py-data-juicer: filename=py_data_juicer-1.0.1-py3-none-any.whl size=331478 sha256=2396c39923ab13a9fe17fe6523734f2ff41b3edb2c97fcc70539a5761e0465f8 2024-12-06T09:55:31,313 Stored in directory: /tmp/pip-ephem-wheel-cache-ms59vp62/wheels/7e/a8/bc/7c44e4643e629fa4243490c97fb8d2e5bf672bcb0244392b01 2024-12-06T09:55:31,333 Successfully built py-data-juicer 2024-12-06T09:55:31,343 Removed build tracker: '/tmp/pip-build-tracker-201tn4cd'