2024-12-20T13:13:46,901 Created temporary directory: /tmp/pip-build-tracker-u3keqqh8 2024-12-20T13:13:46,902 Initialized build tracking at /tmp/pip-build-tracker-u3keqqh8 2024-12-20T13:13:46,903 Created build tracker: /tmp/pip-build-tracker-u3keqqh8 2024-12-20T13:13:46,903 Entered build tracker: /tmp/pip-build-tracker-u3keqqh8 2024-12-20T13:13:46,904 Created temporary directory: /tmp/pip-wheel-fxs_rv_w 2024-12-20T13:13:46,908 Created temporary directory: /tmp/pip-ephem-wheel-cache-pzdykz2i 2024-12-20T13:13:46,952 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2024-12-20T13:13:46,954 2 location(s) to search for versions of py-data-juicer: 2024-12-20T13:13:46,954 * https://pypi.org/simple/py-data-juicer/ 2024-12-20T13:13:46,954 * https://www.piwheels.org/simple/py-data-juicer/ 2024-12-20T13:13:46,955 Fetching project page and analyzing links: https://pypi.org/simple/py-data-juicer/ 2024-12-20T13:13:46,956 Getting page https://pypi.org/simple/py-data-juicer/ 2024-12-20T13:13:46,957 Found index url https://pypi.org/simple/ 2024-12-20T13:13:47,178 Fetched page https://pypi.org/simple/py-data-juicer/ as application/vnd.pypi.simple.v1+json 2024-12-20T13:13:47,181 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/e3/d3/d0724d922e7c55a0664485fdf642124bdcd801df2697e29c9463c4360958/py_data_juicer-0.1.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-20T13:13:47,182 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/fd/c6/e1428310bf534319ddc2e362c6eee21985f097e70cedc0e9cd3e914de826/py_data_juicer-0.1.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-20T13:13:47,183 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/67/70/731e349d2a92bf59a767230b956665dd27ea24a7a948d4a1710154d77a24/py_data_juicer-0.1.2-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-20T13:13:47,183 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/66/08/f584efbdf8277a061ce73c9befc33fec813cbd08020761af2380dd331692/py_data_juicer-0.1.3-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-20T13:13:47,184 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/69/20/9862dfe7a94f10caa0e6387834d1265420613345c7adadd41a3ced7230e8/py_data_juicer-0.2.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-20T13:13:47,185 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/05/34/33fb401d350f1a66cdaefc224419f9d01484722829f5b0c71ef59a8365a4/py_data_juicer-1.0.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-20T13:13:47,185 Found link https://files.pythonhosted.org/packages/49/e9/cfb994255490c36554048e0b3956858e61f5a75ffacfde20849f31f04ef8/py_data_juicer-1.0.0.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.0 2024-12-20T13:13:47,186 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/27/37/150e2198f14349fdd6b647a63feaede809e21dbfc4617aba487f18049828/py_data_juicer-1.0.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-20T13:13:47,187 Found link https://files.pythonhosted.org/packages/4a/60/dadcbe4337a76d8f98022b60d142ac31073d568c9e06a5461b3014d16fc6/py_data_juicer-1.0.1.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.1 2024-12-20T13:13:47,187 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/10/dd/f9cadd6ed2f19c4f94e61a3fd4bb5bb2029c20b2cf61ec86c3a05f89f74b/py_data_juicer-1.0.2-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2024-12-20T13:13:47,188 Found link https://files.pythonhosted.org/packages/95/8f/c0447ce091ca0b86283e8074a3cb350572a5436ca13dceb288887a0de332/py_data_juicer-1.0.2.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.2 2024-12-20T13:13:47,189 Fetching project page and analyzing links: https://www.piwheels.org/simple/py-data-juicer/ 2024-12-20T13:13:47,190 Getting page https://www.piwheels.org/simple/py-data-juicer/ 2024-12-20T13:13:47,191 Found index url https://www.piwheels.org/simple/ 2024-12-20T13:13:47,354 Fetched page https://www.piwheels.org/simple/py-data-juicer/ as text/html 2024-12-20T13:13:47,356 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.1-py3-none-any.whl#sha256=2396c39923ab13a9fe17fe6523734f2ff41b3edb2c97fcc70539a5761e0465f8 (from https://www.piwheels.org/simple/py-data-juicer/) 2024-12-20T13:13:47,357 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.0-py3-none-any.whl#sha256=7cd007456933447474ec9faf5357b741a63e7a0c0dc6230e40bf4498eb1ef554 (from https://www.piwheels.org/simple/py-data-juicer/) 2024-12-20T13:13:47,357 Skipping link: not a file: https://www.piwheels.org/simple/py-data-juicer/ 2024-12-20T13:13:47,358 Skipping link: not a file: https://pypi.org/simple/py-data-juicer/ 2024-12-20T13:13:47,377 Given no hashes to check 1 links for project 'py-data-juicer': discarding no candidates 2024-12-20T13:13:47,378 Collecting py-data-juicer==1.0.2 2024-12-20T13:13:47,381 Created temporary directory: /tmp/pip-unpack-pie1_o5q 2024-12-20T13:13:47,600 Downloading py_data_juicer-1.0.2.tar.gz (245 kB) 2024-12-20T13:13:48,060 Added py-data-juicer==1.0.2 from https://files.pythonhosted.org/packages/95/8f/c0447ce091ca0b86283e8074a3cb350572a5436ca13dceb288887a0de332/py_data_juicer-1.0.2.tar.gz to build tracker '/tmp/pip-build-tracker-u3keqqh8' 2024-12-20T13:13:48,062 Running setup.py (path:/tmp/pip-wheel-fxs_rv_w/py-data-juicer_a0505ae8525f4d9ab47d8fb301381687/setup.py) egg_info for package py-data-juicer 2024-12-20T13:13:48,063 Created temporary directory: /tmp/pip-pip-egg-info-u8lla6_4 2024-12-20T13:13:48,063 Preparing metadata (setup.py): started 2024-12-20T13:13:48,065 Running command python setup.py egg_info 2024-12-20T13:13:48,548 WARNING:root:target file does not exist: environments/minimal_requires.txt 2024-12-20T13:13:48,548 WARNING:root:target file does not exist: environments/science_requires.txt 2024-12-20T13:13:48,549 WARNING:root:target file does not exist: environments/dist_requires.txt 2024-12-20T13:13:48,549 WARNING:root:target file does not exist: environments/dev_requires.txt 2024-12-20T13:13:48,550 WARNING:root:target file does not exist: environments/preprocess_requires.txt 2024-12-20T13:13:48,550 WARNING:root:target file does not exist: environments/quality_classifier_requires.txt 2024-12-20T13:13:48,551 WARNING:root:target file does not exist: environments/sandbox_requires.txt 2024-12-20T13:13:49,160 INFO:root:running egg_info 2024-12-20T13:13:49,189 INFO:root:creating /tmp/pip-pip-egg-info-u8lla6_4/py_data_juicer.egg-info 2024-12-20T13:13:49,190 INFO:root:writing /tmp/pip-pip-egg-info-u8lla6_4/py_data_juicer.egg-info/PKG-INFO 2024-12-20T13:13:49,194 INFO:root:writing dependency_links to /tmp/pip-pip-egg-info-u8lla6_4/py_data_juicer.egg-info/dependency_links.txt 2024-12-20T13:13:49,196 INFO:root:writing entry points to /tmp/pip-pip-egg-info-u8lla6_4/py_data_juicer.egg-info/entry_points.txt 2024-12-20T13:13:49,198 INFO:root:writing requirements to /tmp/pip-pip-egg-info-u8lla6_4/py_data_juicer.egg-info/requires.txt 2024-12-20T13:13:49,199 INFO:root:writing top-level names to /tmp/pip-pip-egg-info-u8lla6_4/py_data_juicer.egg-info/top_level.txt 2024-12-20T13:13:49,200 INFO:root:writing manifest file '/tmp/pip-pip-egg-info-u8lla6_4/py_data_juicer.egg-info/SOURCES.txt' 2024-12-20T13:13:49,305 INFO:root:reading manifest file '/tmp/pip-pip-egg-info-u8lla6_4/py_data_juicer.egg-info/SOURCES.txt' 2024-12-20T13:13:49,307 INFO:root:adding license file 'LICENSE' 2024-12-20T13:13:49,315 INFO:root:writing manifest file '/tmp/pip-pip-egg-info-u8lla6_4/py_data_juicer.egg-info/SOURCES.txt' 2024-12-20T13:13:49,408 Preparing metadata (setup.py): finished with status 'done' 2024-12-20T13:13:49,414 Source in /tmp/pip-wheel-fxs_rv_w/py-data-juicer_a0505ae8525f4d9ab47d8fb301381687 has version 1.0.2, which satisfies requirement py-data-juicer==1.0.2 from https://files.pythonhosted.org/packages/95/8f/c0447ce091ca0b86283e8074a3cb350572a5436ca13dceb288887a0de332/py_data_juicer-1.0.2.tar.gz 2024-12-20T13:13:49,415 Removed py-data-juicer==1.0.2 from https://files.pythonhosted.org/packages/95/8f/c0447ce091ca0b86283e8074a3cb350572a5436ca13dceb288887a0de332/py_data_juicer-1.0.2.tar.gz from build tracker '/tmp/pip-build-tracker-u3keqqh8' 2024-12-20T13:13:49,423 Created temporary directory: /tmp/pip-unpack-ok2rf8ng 2024-12-20T13:13:49,424 Created temporary directory: /tmp/pip-unpack-k65mpqz5 2024-12-20T13:13:49,425 Building wheels for collected packages: py-data-juicer 2024-12-20T13:13:49,428 Created temporary directory: /tmp/pip-wheel-7kzhu2dm 2024-12-20T13:13:49,429 Building wheel for py-data-juicer (setup.py): started 2024-12-20T13:13:49,430 Destination directory: /tmp/pip-wheel-7kzhu2dm 2024-12-20T13:13:49,431 Running command python setup.py bdist_wheel 2024-12-20T13:13:49,879 WARNING:root:target file does not exist: environments/minimal_requires.txt 2024-12-20T13:13:49,879 WARNING:root:target file does not exist: environments/science_requires.txt 2024-12-20T13:13:49,880 WARNING:root:target file does not exist: environments/dist_requires.txt 2024-12-20T13:13:49,881 WARNING:root:target file does not exist: environments/dev_requires.txt 2024-12-20T13:13:49,881 WARNING:root:target file does not exist: environments/preprocess_requires.txt 2024-12-20T13:13:49,882 WARNING:root:target file does not exist: environments/quality_classifier_requires.txt 2024-12-20T13:13:49,882 WARNING:root:target file does not exist: environments/sandbox_requires.txt 2024-12-20T13:13:50,447 INFO:root:running bdist_wheel 2024-12-20T13:13:50,579 INFO:root:running build 2024-12-20T13:13:50,580 INFO:root:running build_py 2024-12-20T13:13:50,610 INFO:root:creating build/lib/data_juicer 2024-12-20T13:13:50,613 INFO:root:copying data_juicer/__init__.py -> build/lib/data_juicer 2024-12-20T13:13:50,615 INFO:root:creating build/lib/data_juicer/config 2024-12-20T13:13:50,616 INFO:root:copying data_juicer/config/__init__.py -> build/lib/data_juicer/config 2024-12-20T13:13:50,618 INFO:root:copying data_juicer/config/config.py -> build/lib/data_juicer/config 2024-12-20T13:13:50,621 INFO:root:creating build/lib/data_juicer/ops 2024-12-20T13:13:50,622 INFO:root:copying data_juicer/ops/__init__.py -> build/lib/data_juicer/ops 2024-12-20T13:13:50,624 INFO:root:copying data_juicer/ops/op_fusion.py -> build/lib/data_juicer/ops 2024-12-20T13:13:50,627 INFO:root:copying data_juicer/ops/load.py -> build/lib/data_juicer/ops 2024-12-20T13:13:50,628 INFO:root:copying data_juicer/ops/base_op.py -> build/lib/data_juicer/ops 2024-12-20T13:13:50,632 INFO:root:creating build/lib/data_juicer/utils 2024-12-20T13:13:50,633 INFO:root:copying data_juicer/utils/registry.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,635 INFO:root:copying data_juicer/utils/compress.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,638 INFO:root:copying data_juicer/utils/process_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,640 INFO:root:copying data_juicer/utils/common_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,642 INFO:root:copying data_juicer/utils/mm_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,645 INFO:root:copying data_juicer/utils/cache_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,646 INFO:root:copying data_juicer/utils/__init__.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,648 INFO:root:copying data_juicer/utils/logger_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,650 INFO:root:copying data_juicer/utils/asset_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,652 INFO:root:copying data_juicer/utils/auto_install_mapping.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,655 INFO:root:copying data_juicer/utils/fingerprint_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,657 INFO:root:copying data_juicer/utils/unittest_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,660 INFO:root:copying data_juicer/utils/resource_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,661 INFO:root:copying data_juicer/utils/constant.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,664 INFO:root:copying data_juicer/utils/file_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,666 INFO:root:copying data_juicer/utils/model_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,669 INFO:root:copying data_juicer/utils/lazy_loader.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,670 INFO:root:copying data_juicer/utils/auto_install_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,672 INFO:root:copying data_juicer/utils/availability_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,674 INFO:root:copying data_juicer/utils/ckpt_utils.py -> build/lib/data_juicer/utils 2024-12-20T13:13:50,677 INFO:root:creating build/lib/data_juicer/analysis 2024-12-20T13:13:50,678 INFO:root:copying data_juicer/analysis/collector.py -> build/lib/data_juicer/analysis 2024-12-20T13:13:50,679 INFO:root:copying data_juicer/analysis/measure.py -> build/lib/data_juicer/analysis 2024-12-20T13:13:50,682 INFO:root:copying data_juicer/analysis/__init__.py -> build/lib/data_juicer/analysis 2024-12-20T13:13:50,683 INFO:root:copying data_juicer/analysis/draw.py -> build/lib/data_juicer/analysis 2024-12-20T13:13:50,685 INFO:root:copying data_juicer/analysis/overall_analysis.py -> build/lib/data_juicer/analysis 2024-12-20T13:13:50,687 INFO:root:copying data_juicer/analysis/column_wise_analysis.py -> build/lib/data_juicer/analysis 2024-12-20T13:13:50,690 INFO:root:copying data_juicer/analysis/diversity_analysis.py -> build/lib/data_juicer/analysis 2024-12-20T13:13:50,693 INFO:root:creating build/lib/data_juicer/core 2024-12-20T13:13:50,694 INFO:root:copying data_juicer/core/analyzer.py -> build/lib/data_juicer/core 2024-12-20T13:13:50,697 INFO:root:copying data_juicer/core/executor.py -> build/lib/data_juicer/core 2024-12-20T13:13:50,699 INFO:root:copying data_juicer/core/__init__.py -> build/lib/data_juicer/core 2024-12-20T13:13:50,701 INFO:root:copying data_juicer/core/data.py -> build/lib/data_juicer/core 2024-12-20T13:13:50,704 INFO:root:copying data_juicer/core/exporter.py -> build/lib/data_juicer/core 2024-12-20T13:13:50,707 INFO:root:copying data_juicer/core/ray_executor.py -> build/lib/data_juicer/core 2024-12-20T13:13:50,709 INFO:root:copying data_juicer/core/adapter.py -> build/lib/data_juicer/core 2024-12-20T13:13:50,712 INFO:root:copying data_juicer/core/tracer.py -> build/lib/data_juicer/core 2024-12-20T13:13:50,715 INFO:root:copying data_juicer/core/monitor.py -> build/lib/data_juicer/core 2024-12-20T13:13:50,718 INFO:root:copying data_juicer/core/ray_data.py -> build/lib/data_juicer/core 2024-12-20T13:13:50,721 INFO:root:creating build/lib/data_juicer/format 2024-12-20T13:13:50,722 INFO:root:copying data_juicer/format/csv_formatter.py -> build/lib/data_juicer/format 2024-12-20T13:13:50,725 INFO:root:copying data_juicer/format/__init__.py -> build/lib/data_juicer/format 2024-12-20T13:13:50,727 INFO:root:copying data_juicer/format/tsv_formatter.py -> build/lib/data_juicer/format 2024-12-20T13:13:50,729 INFO:root:copying data_juicer/format/parquet_formatter.py -> build/lib/data_juicer/format 2024-12-20T13:13:50,732 INFO:root:copying data_juicer/format/empty_formatter.py -> build/lib/data_juicer/format 2024-12-20T13:13:50,734 INFO:root:copying data_juicer/format/json_formatter.py -> build/lib/data_juicer/format 2024-12-20T13:13:50,736 INFO:root:copying data_juicer/format/load.py -> build/lib/data_juicer/format 2024-12-20T13:13:50,738 INFO:root:copying data_juicer/format/formatter.py -> build/lib/data_juicer/format 2024-12-20T13:13:50,741 INFO:root:copying data_juicer/format/text_formatter.py -> build/lib/data_juicer/format 2024-12-20T13:13:50,744 INFO:root:copying data_juicer/format/mixture_formatter.py -> build/lib/data_juicer/format 2024-12-20T13:13:50,747 INFO:root:creating build/lib/data_juicer/ops/common 2024-12-20T13:13:50,748 INFO:root:copying data_juicer/ops/common/__init__.py -> build/lib/data_juicer/ops/common 2024-12-20T13:13:50,757 INFO:root:copying data_juicer/ops/common/special_characters.py -> build/lib/data_juicer/ops/common 2024-12-20T13:13:50,759 INFO:root:copying data_juicer/ops/common/helper_func.py -> build/lib/data_juicer/ops/common 2024-12-20T13:13:50,762 INFO:root:creating build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,763 INFO:root:copying data_juicer/ops/deduplicator/ray_document_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,764 INFO:root:copying data_juicer/ops/deduplicator/ray_video_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,766 INFO:root:copying data_juicer/ops/deduplicator/__init__.py -> build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,768 INFO:root:copying data_juicer/ops/deduplicator/document_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,770 INFO:root:copying data_juicer/ops/deduplicator/document_simhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,772 INFO:root:copying data_juicer/ops/deduplicator/document_minhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,774 INFO:root:copying data_juicer/ops/deduplicator/ray_image_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,776 INFO:root:copying data_juicer/ops/deduplicator/image_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,778 INFO:root:copying data_juicer/ops/deduplicator/ray_basic_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,780 INFO:root:copying data_juicer/ops/deduplicator/video_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2024-12-20T13:13:50,783 INFO:root:creating build/lib/data_juicer/ops/aggregator 2024-12-20T13:13:50,784 INFO:root:copying data_juicer/ops/aggregator/__init__.py -> build/lib/data_juicer/ops/aggregator 2024-12-20T13:13:50,786 INFO:root:copying data_juicer/ops/aggregator/most_relavant_entities_aggregator.py -> build/lib/data_juicer/ops/aggregator 2024-12-20T13:13:50,788 INFO:root:copying data_juicer/ops/aggregator/entity_attribute_aggregator.py -> build/lib/data_juicer/ops/aggregator 2024-12-20T13:13:50,790 INFO:root:copying data_juicer/ops/aggregator/nested_aggregator.py -> build/lib/data_juicer/ops/aggregator 2024-12-20T13:13:50,795 INFO:root:creating build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,796 INFO:root:copying data_juicer/ops/mapper/image_captioning_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,799 INFO:root:copying data_juicer/ops/mapper/extract_event_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,801 INFO:root:copying data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,802 INFO:root:copying data_juicer/ops/mapper/calibrate_qa_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,805 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,808 INFO:root:copying data_juicer/ops/mapper/punctuation_normalization_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,810 INFO:root:copying data_juicer/ops/mapper/clean_links_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,811 INFO:root:copying data_juicer/ops/mapper/expand_macro_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,814 INFO:root:copying data_juicer/ops/mapper/replace_content_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,815 INFO:root:copying data_juicer/ops/mapper/extract_support_text_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,818 INFO:root:copying data_juicer/ops/mapper/optimize_response_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,819 INFO:root:copying data_juicer/ops/mapper/__init__.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,822 INFO:root:copying data_juicer/ops/mapper/clean_html_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,823 INFO:root:copying data_juicer/ops/mapper/fix_unicode_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,825 INFO:root:copying data_juicer/ops/mapper/optimize_qa_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,827 INFO:root:copying data_juicer/ops/mapper/video_extract_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,829 INFO:root:copying data_juicer/ops/mapper/video_split_by_scene_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,831 INFO:root:copying data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,834 INFO:root:copying data_juicer/ops/mapper/remove_table_text_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,835 INFO:root:copying data_juicer/ops/mapper/optimize_query_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,837 INFO:root:copying data_juicer/ops/mapper/remove_long_words_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,839 INFO:root:copying data_juicer/ops/mapper/remove_bibliography_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,841 INFO:root:copying data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,843 INFO:root:copying data_juicer/ops/mapper/clean_email_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,845 INFO:root:copying data_juicer/ops/mapper/nlpaug_en_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,847 INFO:root:copying data_juicer/ops/mapper/remove_header_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,849 INFO:root:copying data_juicer/ops/mapper/remove_specific_chars_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,850 INFO:root:copying data_juicer/ops/mapper/image_tagging_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,852 INFO:root:copying data_juicer/ops/mapper/extract_keyword_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,855 INFO:root:copying data_juicer/ops/mapper/extract_entity_attribute_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,857 INFO:root:copying data_juicer/ops/mapper/image_diffusion_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,859 INFO:root:copying data_juicer/ops/mapper/sentence_split_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,861 INFO:root:copying data_juicer/ops/mapper/clean_copyright_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,863 INFO:root:copying data_juicer/ops/mapper/calibrate_response_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,865 INFO:root:copying data_juicer/ops/mapper/video_resize_resolution_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,867 INFO:root:copying data_juicer/ops/mapper/generate_qa_from_text_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,869 INFO:root:copying data_juicer/ops/mapper/chinese_convert_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,871 INFO:root:copying data_juicer/ops/mapper/clean_ip_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,873 INFO:root:copying data_juicer/ops/mapper/text_chunk_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,875 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_video_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,877 INFO:root:copying data_juicer/ops/mapper/generate_qa_from_examples_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,879 INFO:root:copying data_juicer/ops/mapper/video_split_by_duration_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,882 INFO:root:copying data_juicer/ops/mapper/remove_non_chinese_character_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,883 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_audio_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,885 INFO:root:copying data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,888 INFO:root:copying data_juicer/ops/mapper/image_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,890 INFO:root:copying data_juicer/ops/mapper/video_face_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,892 INFO:root:copying data_juicer/ops/mapper/video_split_by_key_frame_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,894 INFO:root:copying data_juicer/ops/mapper/python_lambda_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,896 INFO:root:copying data_juicer/ops/mapper/remove_repeat_sentences_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,898 INFO:root:copying data_juicer/ops/mapper/video_tagging_from_audio_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,900 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,902 INFO:root:copying data_juicer/ops/mapper/relation_identity_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,904 INFO:root:copying data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,906 INFO:root:copying data_juicer/ops/mapper/extract_nickname_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,908 INFO:root:copying data_juicer/ops/mapper/calibrate_query_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,910 INFO:root:copying data_juicer/ops/mapper/image_face_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,912 INFO:root:copying data_juicer/ops/mapper/whitespace_normalization_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,914 INFO:root:copying data_juicer/ops/mapper/python_file_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,916 INFO:root:copying data_juicer/ops/mapper/video_remove_watermark_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,918 INFO:root:copying data_juicer/ops/mapper/nlpcda_zh_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,921 INFO:root:copying data_juicer/ops/mapper/remove_comments_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,922 INFO:root:copying data_juicer/ops/mapper/video_tagging_from_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,925 INFO:root:copying data_juicer/ops/mapper/extract_entity_relation_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,927 INFO:root:copying data_juicer/ops/mapper/pair_preference_mapper.py -> build/lib/data_juicer/ops/mapper 2024-12-20T13:13:50,930 INFO:root:creating build/lib/data_juicer/ops/grouper 2024-12-20T13:13:50,931 INFO:root:copying data_juicer/ops/grouper/__init__.py -> build/lib/data_juicer/ops/grouper 2024-12-20T13:13:50,933 INFO:root:copying data_juicer/ops/grouper/key_value_grouper.py -> build/lib/data_juicer/ops/grouper 2024-12-20T13:13:50,935 INFO:root:copying data_juicer/ops/grouper/naive_grouper.py -> build/lib/data_juicer/ops/grouper 2024-12-20T13:13:50,937 INFO:root:creating build/lib/data_juicer/ops/selector 2024-12-20T13:13:50,938 INFO:root:copying data_juicer/ops/selector/__init__.py -> build/lib/data_juicer/ops/selector 2024-12-20T13:13:50,941 INFO:root:copying data_juicer/ops/selector/topk_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2024-12-20T13:13:50,942 INFO:root:copying data_juicer/ops/selector/random_selector.py -> build/lib/data_juicer/ops/selector 2024-12-20T13:13:50,945 INFO:root:copying data_juicer/ops/selector/range_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2024-12-20T13:13:50,947 INFO:root:copying data_juicer/ops/selector/frequency_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2024-12-20T13:13:50,950 INFO:root:creating build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,951 INFO:root:copying data_juicer/ops/filter/text_entity_dependency_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,953 INFO:root:copying data_juicer/ops/filter/image_aesthetics_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,955 INFO:root:copying data_juicer/ops/filter/image_text_similarity_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,957 INFO:root:copying data_juicer/ops/filter/average_line_length_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,959 INFO:root:copying data_juicer/ops/filter/special_characters_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,961 INFO:root:copying data_juicer/ops/filter/language_id_score_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,963 INFO:root:copying data_juicer/ops/filter/text_action_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,965 INFO:root:copying data_juicer/ops/filter/video_watermark_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,967 INFO:root:copying data_juicer/ops/filter/audio_nmf_snr_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,969 INFO:root:copying data_juicer/ops/filter/token_num_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,971 INFO:root:copying data_juicer/ops/filter/flagged_words_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,973 INFO:root:copying data_juicer/ops/filter/__init__.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,975 INFO:root:copying data_juicer/ops/filter/audio_size_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,977 INFO:root:copying data_juicer/ops/filter/word_repetition_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,979 INFO:root:copying data_juicer/ops/filter/words_num_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,981 INFO:root:copying data_juicer/ops/filter/alphanumeric_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,983 INFO:root:copying data_juicer/ops/filter/video_frames_text_similarity_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,985 INFO:root:copying data_juicer/ops/filter/image_size_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,987 INFO:root:copying data_juicer/ops/filter/image_nsfw_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,989 INFO:root:copying data_juicer/ops/filter/audio_duration_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,991 INFO:root:copying data_juicer/ops/filter/image_aspect_ratio_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,993 INFO:root:copying data_juicer/ops/filter/stopwords_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,995 INFO:root:copying data_juicer/ops/filter/text_length_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,997 INFO:root:copying data_juicer/ops/filter/video_tagging_from_frames_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:50,999 INFO:root:copying data_juicer/ops/filter/suffix_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,001 INFO:root:copying data_juicer/ops/filter/image_face_count_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,003 INFO:root:copying data_juicer/ops/filter/image_watermark_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,006 INFO:root:copying data_juicer/ops/filter/video_nsfw_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,008 INFO:root:copying data_juicer/ops/filter/image_text_matching_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,010 INFO:root:copying data_juicer/ops/filter/video_ocr_area_ratio_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,013 INFO:root:copying data_juicer/ops/filter/video_motion_score_raft_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,014 INFO:root:copying data_juicer/ops/filter/maximum_line_length_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,017 INFO:root:copying data_juicer/ops/filter/character_repetition_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,018 INFO:root:copying data_juicer/ops/filter/specified_numeric_field_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,020 INFO:root:copying data_juicer/ops/filter/video_aesthetics_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,022 INFO:root:copying data_juicer/ops/filter/video_motion_score_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,025 INFO:root:copying data_juicer/ops/filter/video_resolution_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,026 INFO:root:copying data_juicer/ops/filter/specified_field_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,028 INFO:root:copying data_juicer/ops/filter/image_pair_similarity_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,030 INFO:root:copying data_juicer/ops/filter/image_shape_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,032 INFO:root:copying data_juicer/ops/filter/video_aspect_ratio_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,034 INFO:root:copying data_juicer/ops/filter/perplexity_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,036 INFO:root:copying data_juicer/ops/filter/video_duration_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,038 INFO:root:copying data_juicer/ops/filter/phrase_grounding_recall_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,040 INFO:root:copying data_juicer/ops/filter/image_face_ratio_filter.py -> build/lib/data_juicer/ops/filter 2024-12-20T13:13:51,043 INFO:root:creating build/lib/data_juicer/tools 2024-12-20T13:13:51,044 INFO:root:copying tools/__init__.py -> build/lib/data_juicer/tools 2024-12-20T13:13:51,046 INFO:root:copying tools/sandbox_starter.py -> build/lib/data_juicer/tools 2024-12-20T13:13:51,048 INFO:root:copying tools/dj_install.py -> build/lib/data_juicer/tools 2024-12-20T13:13:51,050 INFO:root:copying tools/process_data.py -> build/lib/data_juicer/tools 2024-12-20T13:13:51,051 INFO:root:copying tools/analyze_data.py -> build/lib/data_juicer/tools 2024-12-20T13:13:51,101 /usr/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. 2024-12-20T13:13:51,101 !! 2024-12-20T13:13:51,103 ******************************************************************************** 2024-12-20T13:13:51,103 Please avoid running ``setup.py`` directly. 2024-12-20T13:13:51,104 Instead, use pypa/build, pypa/installer or other 2024-12-20T13:13:51,105 standards-based tools. 2024-12-20T13:13:51,106 See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. 2024-12-20T13:13:51,106 ******************************************************************************** 2024-12-20T13:13:51,107 !! 2024-12-20T13:13:51,108 self.initialize_options() 2024-12-20T13:13:51,129 INFO:root:installing to build/bdist.linux-armv7l/wheel 2024-12-20T13:13:51,129 INFO:root:running install 2024-12-20T13:13:51,154 INFO:root:running install_lib 2024-12-20T13:13:51,181 INFO:root:creating build/bdist.linux-armv7l/wheel 2024-12-20T13:13:51,183 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer 2024-12-20T13:13:51,185 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/config 2024-12-20T13:13:51,186 INFO:root:copying build/lib/data_juicer/config/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/config 2024-12-20T13:13:51,188 INFO:root:copying build/lib/data_juicer/config/config.py -> build/bdist.linux-armv7l/wheel/./data_juicer/config 2024-12-20T13:13:51,192 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops 2024-12-20T13:13:51,193 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/common 2024-12-20T13:13:51,194 INFO:root:copying build/lib/data_juicer/ops/common/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2024-12-20T13:13:51,196 INFO:root:copying build/lib/data_juicer/ops/common/special_characters.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2024-12-20T13:13:51,198 INFO:root:copying build/lib/data_juicer/ops/common/helper_func.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2024-12-20T13:13:51,200 INFO:root:copying build/lib/data_juicer/ops/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2024-12-20T13:13:51,202 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/deduplicator 2024-12-20T13:13:51,203 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_document_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-20T13:13:51,205 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_video_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-20T13:13:51,207 INFO:root:copying build/lib/data_juicer/ops/deduplicator/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-20T13:13:51,209 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-20T13:13:51,211 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_simhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-20T13:13:51,213 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_minhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-20T13:13:51,216 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_image_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-20T13:13:51,218 INFO:root:copying build/lib/data_juicer/ops/deduplicator/image_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-20T13:13:51,220 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_basic_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-20T13:13:51,222 INFO:root:copying build/lib/data_juicer/ops/deduplicator/video_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2024-12-20T13:13:51,224 INFO:root:copying build/lib/data_juicer/ops/op_fusion.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2024-12-20T13:13:51,227 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/aggregator 2024-12-20T13:13:51,228 INFO:root:copying build/lib/data_juicer/ops/aggregator/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2024-12-20T13:13:51,230 INFO:root:copying build/lib/data_juicer/ops/aggregator/most_relavant_entities_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2024-12-20T13:13:51,232 INFO:root:copying build/lib/data_juicer/ops/aggregator/entity_attribute_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2024-12-20T13:13:51,235 INFO:root:copying build/lib/data_juicer/ops/aggregator/nested_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2024-12-20T13:13:51,238 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/mapper 2024-12-20T13:13:51,239 INFO:root:copying build/lib/data_juicer/ops/mapper/image_captioning_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,242 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_event_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,244 INFO:root:copying build/lib/data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,246 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_qa_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,248 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,250 INFO:root:copying build/lib/data_juicer/ops/mapper/punctuation_normalization_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,252 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_links_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,254 INFO:root:copying build/lib/data_juicer/ops/mapper/expand_macro_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,256 INFO:root:copying build/lib/data_juicer/ops/mapper/replace_content_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,257 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_support_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,260 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_response_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,261 INFO:root:copying build/lib/data_juicer/ops/mapper/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,263 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_html_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,265 INFO:root:copying build/lib/data_juicer/ops/mapper/fix_unicode_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,267 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_qa_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,269 INFO:root:copying build/lib/data_juicer/ops/mapper/video_extract_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,271 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_scene_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,273 INFO:root:copying build/lib/data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,276 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_table_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,277 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_query_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,279 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_long_words_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,281 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_bibliography_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,283 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,285 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_email_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,286 INFO:root:copying build/lib/data_juicer/ops/mapper/nlpaug_en_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,289 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_header_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,290 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_specific_chars_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,292 INFO:root:copying build/lib/data_juicer/ops/mapper/image_tagging_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,294 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_keyword_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,296 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_entity_attribute_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,298 INFO:root:copying build/lib/data_juicer/ops/mapper/image_diffusion_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,300 INFO:root:copying build/lib/data_juicer/ops/mapper/sentence_split_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,302 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_copyright_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,304 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_response_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,306 INFO:root:copying build/lib/data_juicer/ops/mapper/video_resize_resolution_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,308 INFO:root:copying build/lib/data_juicer/ops/mapper/generate_qa_from_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,310 INFO:root:copying build/lib/data_juicer/ops/mapper/chinese_convert_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,312 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_ip_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,314 INFO:root:copying build/lib/data_juicer/ops/mapper/text_chunk_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,316 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_video_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,318 INFO:root:copying build/lib/data_juicer/ops/mapper/generate_qa_from_examples_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,321 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_duration_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,323 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_non_chinese_character_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,325 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_audio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,327 INFO:root:copying build/lib/data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,330 INFO:root:copying build/lib/data_juicer/ops/mapper/image_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,331 INFO:root:copying build/lib/data_juicer/ops/mapper/video_face_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,333 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_key_frame_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,336 INFO:root:copying build/lib/data_juicer/ops/mapper/python_lambda_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,337 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_repeat_sentences_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,339 INFO:root:copying build/lib/data_juicer/ops/mapper/video_tagging_from_audio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,341 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,343 INFO:root:copying build/lib/data_juicer/ops/mapper/relation_identity_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,346 INFO:root:copying build/lib/data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,348 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_nickname_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,350 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_query_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,351 INFO:root:copying build/lib/data_juicer/ops/mapper/image_face_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,353 INFO:root:copying build/lib/data_juicer/ops/mapper/whitespace_normalization_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,355 INFO:root:copying build/lib/data_juicer/ops/mapper/python_file_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,357 INFO:root:copying build/lib/data_juicer/ops/mapper/video_remove_watermark_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,359 INFO:root:copying build/lib/data_juicer/ops/mapper/nlpcda_zh_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,361 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_comments_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,363 INFO:root:copying build/lib/data_juicer/ops/mapper/video_tagging_from_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,365 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_entity_relation_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,367 INFO:root:copying build/lib/data_juicer/ops/mapper/pair_preference_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2024-12-20T13:13:51,370 INFO:root:copying build/lib/data_juicer/ops/load.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2024-12-20T13:13:51,371 INFO:root:copying build/lib/data_juicer/ops/base_op.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2024-12-20T13:13:51,374 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/grouper 2024-12-20T13:13:51,375 INFO:root:copying build/lib/data_juicer/ops/grouper/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2024-12-20T13:13:51,377 INFO:root:copying build/lib/data_juicer/ops/grouper/key_value_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2024-12-20T13:13:51,379 INFO:root:copying build/lib/data_juicer/ops/grouper/naive_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2024-12-20T13:13:51,381 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/selector 2024-12-20T13:13:51,382 INFO:root:copying build/lib/data_juicer/ops/selector/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2024-12-20T13:13:51,384 INFO:root:copying build/lib/data_juicer/ops/selector/topk_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2024-12-20T13:13:51,386 INFO:root:copying build/lib/data_juicer/ops/selector/random_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2024-12-20T13:13:51,388 INFO:root:copying build/lib/data_juicer/ops/selector/range_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2024-12-20T13:13:51,390 INFO:root:copying build/lib/data_juicer/ops/selector/frequency_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2024-12-20T13:13:51,393 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/filter 2024-12-20T13:13:51,394 INFO:root:copying build/lib/data_juicer/ops/filter/text_entity_dependency_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,396 INFO:root:copying build/lib/data_juicer/ops/filter/image_aesthetics_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,399 INFO:root:copying build/lib/data_juicer/ops/filter/image_text_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,401 INFO:root:copying build/lib/data_juicer/ops/filter/average_line_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,403 INFO:root:copying build/lib/data_juicer/ops/filter/special_characters_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,405 INFO:root:copying build/lib/data_juicer/ops/filter/language_id_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,406 INFO:root:copying build/lib/data_juicer/ops/filter/text_action_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,408 INFO:root:copying build/lib/data_juicer/ops/filter/video_watermark_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,410 INFO:root:copying build/lib/data_juicer/ops/filter/audio_nmf_snr_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,412 INFO:root:copying build/lib/data_juicer/ops/filter/token_num_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,414 INFO:root:copying build/lib/data_juicer/ops/filter/flagged_words_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,416 INFO:root:copying build/lib/data_juicer/ops/filter/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,418 INFO:root:copying build/lib/data_juicer/ops/filter/audio_size_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,420 INFO:root:copying build/lib/data_juicer/ops/filter/word_repetition_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,422 INFO:root:copying build/lib/data_juicer/ops/filter/words_num_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,424 INFO:root:copying build/lib/data_juicer/ops/filter/alphanumeric_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,426 INFO:root:copying build/lib/data_juicer/ops/filter/video_frames_text_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,428 INFO:root:copying build/lib/data_juicer/ops/filter/image_size_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,430 INFO:root:copying build/lib/data_juicer/ops/filter/image_nsfw_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,432 INFO:root:copying build/lib/data_juicer/ops/filter/audio_duration_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,434 INFO:root:copying build/lib/data_juicer/ops/filter/image_aspect_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,436 INFO:root:copying build/lib/data_juicer/ops/filter/stopwords_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,438 INFO:root:copying build/lib/data_juicer/ops/filter/text_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,440 INFO:root:copying build/lib/data_juicer/ops/filter/video_tagging_from_frames_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,442 INFO:root:copying build/lib/data_juicer/ops/filter/suffix_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,444 INFO:root:copying build/lib/data_juicer/ops/filter/image_face_count_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,446 INFO:root:copying build/lib/data_juicer/ops/filter/image_watermark_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,448 INFO:root:copying build/lib/data_juicer/ops/filter/video_nsfw_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,450 INFO:root:copying build/lib/data_juicer/ops/filter/image_text_matching_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,452 INFO:root:copying build/lib/data_juicer/ops/filter/video_ocr_area_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,454 INFO:root:copying build/lib/data_juicer/ops/filter/video_motion_score_raft_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,456 INFO:root:copying build/lib/data_juicer/ops/filter/maximum_line_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,458 INFO:root:copying build/lib/data_juicer/ops/filter/character_repetition_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,459 INFO:root:copying build/lib/data_juicer/ops/filter/specified_numeric_field_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,461 INFO:root:copying build/lib/data_juicer/ops/filter/video_aesthetics_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,463 INFO:root:copying build/lib/data_juicer/ops/filter/video_motion_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,465 INFO:root:copying build/lib/data_juicer/ops/filter/video_resolution_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,467 INFO:root:copying build/lib/data_juicer/ops/filter/specified_field_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,469 INFO:root:copying build/lib/data_juicer/ops/filter/image_pair_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,471 INFO:root:copying build/lib/data_juicer/ops/filter/image_shape_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,473 INFO:root:copying build/lib/data_juicer/ops/filter/video_aspect_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,475 INFO:root:copying build/lib/data_juicer/ops/filter/perplexity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,477 INFO:root:copying build/lib/data_juicer/ops/filter/video_duration_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,479 INFO:root:copying build/lib/data_juicer/ops/filter/phrase_grounding_recall_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,481 INFO:root:copying build/lib/data_juicer/ops/filter/image_face_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2024-12-20T13:13:51,484 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/tools 2024-12-20T13:13:51,485 INFO:root:copying build/lib/data_juicer/tools/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2024-12-20T13:13:51,486 INFO:root:copying build/lib/data_juicer/tools/sandbox_starter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2024-12-20T13:13:51,488 INFO:root:copying build/lib/data_juicer/tools/dj_install.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2024-12-20T13:13:51,490 INFO:root:copying build/lib/data_juicer/tools/process_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2024-12-20T13:13:51,492 INFO:root:copying build/lib/data_juicer/tools/analyze_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2024-12-20T13:13:51,494 INFO:root:copying build/lib/data_juicer/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer 2024-12-20T13:13:51,496 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/utils 2024-12-20T13:13:51,497 INFO:root:copying build/lib/data_juicer/utils/registry.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,499 INFO:root:copying build/lib/data_juicer/utils/compress.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,502 INFO:root:copying build/lib/data_juicer/utils/process_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,503 INFO:root:copying build/lib/data_juicer/utils/common_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,505 INFO:root:copying build/lib/data_juicer/utils/mm_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,508 INFO:root:copying build/lib/data_juicer/utils/cache_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,510 INFO:root:copying build/lib/data_juicer/utils/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,511 INFO:root:copying build/lib/data_juicer/utils/logger_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,513 INFO:root:copying build/lib/data_juicer/utils/asset_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,515 INFO:root:copying build/lib/data_juicer/utils/auto_install_mapping.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,517 INFO:root:copying build/lib/data_juicer/utils/fingerprint_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,519 INFO:root:copying build/lib/data_juicer/utils/unittest_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,522 INFO:root:copying build/lib/data_juicer/utils/resource_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,523 INFO:root:copying build/lib/data_juicer/utils/constant.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,526 INFO:root:copying build/lib/data_juicer/utils/file_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,528 INFO:root:copying build/lib/data_juicer/utils/model_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,531 INFO:root:copying build/lib/data_juicer/utils/lazy_loader.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,532 INFO:root:copying build/lib/data_juicer/utils/auto_install_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,534 INFO:root:copying build/lib/data_juicer/utils/availability_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,536 INFO:root:copying build/lib/data_juicer/utils/ckpt_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2024-12-20T13:13:51,539 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/analysis 2024-12-20T13:13:51,540 INFO:root:copying build/lib/data_juicer/analysis/collector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-20T13:13:51,542 INFO:root:copying build/lib/data_juicer/analysis/measure.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-20T13:13:51,545 INFO:root:copying build/lib/data_juicer/analysis/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-20T13:13:51,546 INFO:root:copying build/lib/data_juicer/analysis/draw.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-20T13:13:51,548 INFO:root:copying build/lib/data_juicer/analysis/overall_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-20T13:13:51,550 INFO:root:copying build/lib/data_juicer/analysis/column_wise_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-20T13:13:51,552 INFO:root:copying build/lib/data_juicer/analysis/diversity_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2024-12-20T13:13:51,555 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/core 2024-12-20T13:13:51,556 INFO:root:copying build/lib/data_juicer/core/analyzer.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-20T13:13:51,558 INFO:root:copying build/lib/data_juicer/core/executor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-20T13:13:51,561 INFO:root:copying build/lib/data_juicer/core/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-20T13:13:51,562 INFO:root:copying build/lib/data_juicer/core/data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-20T13:13:51,565 INFO:root:copying build/lib/data_juicer/core/exporter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-20T13:13:51,567 INFO:root:copying build/lib/data_juicer/core/ray_executor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-20T13:13:51,568 INFO:root:copying build/lib/data_juicer/core/adapter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-20T13:13:51,571 INFO:root:copying build/lib/data_juicer/core/tracer.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-20T13:13:51,573 INFO:root:copying build/lib/data_juicer/core/monitor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-20T13:13:51,575 INFO:root:copying build/lib/data_juicer/core/ray_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2024-12-20T13:13:51,578 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/format 2024-12-20T13:13:51,579 INFO:root:copying build/lib/data_juicer/format/csv_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-20T13:13:51,581 INFO:root:copying build/lib/data_juicer/format/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-20T13:13:51,583 INFO:root:copying build/lib/data_juicer/format/tsv_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-20T13:13:51,585 INFO:root:copying build/lib/data_juicer/format/parquet_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-20T13:13:51,587 INFO:root:copying build/lib/data_juicer/format/empty_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-20T13:13:51,589 INFO:root:copying build/lib/data_juicer/format/json_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-20T13:13:51,590 INFO:root:copying build/lib/data_juicer/format/load.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-20T13:13:51,592 INFO:root:copying build/lib/data_juicer/format/formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-20T13:13:51,594 INFO:root:copying build/lib/data_juicer/format/text_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-20T13:13:51,597 INFO:root:copying build/lib/data_juicer/format/mixture_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2024-12-20T13:13:51,599 INFO:root:running install_egg_info 2024-12-20T13:13:51,633 INFO:root:running egg_info 2024-12-20T13:13:51,660 INFO:root:writing py_data_juicer.egg-info/PKG-INFO 2024-12-20T13:13:51,664 INFO:root:writing dependency_links to py_data_juicer.egg-info/dependency_links.txt 2024-12-20T13:13:51,666 INFO:root:writing entry points to py_data_juicer.egg-info/entry_points.txt 2024-12-20T13:13:51,667 INFO:root:writing requirements to py_data_juicer.egg-info/requires.txt 2024-12-20T13:13:51,668 INFO:root:writing top-level names to py_data_juicer.egg-info/top_level.txt 2024-12-20T13:13:51,715 INFO:root:reading manifest file 'py_data_juicer.egg-info/SOURCES.txt' 2024-12-20T13:13:51,727 INFO:root:adding license file 'LICENSE' 2024-12-20T13:13:51,737 INFO:root:writing manifest file 'py_data_juicer.egg-info/SOURCES.txt' 2024-12-20T13:13:51,739 INFO:root:Copying py_data_juicer.egg-info to build/bdist.linux-armv7l/wheel/./py_data_juicer-1.0.2-py3.11.egg-info 2024-12-20T13:13:51,752 INFO:root:running install_scripts 2024-12-20T13:13:51,771 INFO:root:creating build/bdist.linux-armv7l/wheel/py_data_juicer-1.0.2.dist-info/WHEEL 2024-12-20T13:13:51,773 INFO:wheel:creating '/tmp/pip-wheel-7kzhu2dm/py_data_juicer-1.0.2-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2024-12-20T13:13:51,776 INFO:wheel:adding 'data_juicer/__init__.py' 2024-12-20T13:13:51,778 INFO:wheel:adding 'data_juicer/analysis/__init__.py' 2024-12-20T13:13:51,779 INFO:wheel:adding 'data_juicer/analysis/collector.py' 2024-12-20T13:13:51,781 INFO:wheel:adding 'data_juicer/analysis/column_wise_analysis.py' 2024-12-20T13:13:51,782 INFO:wheel:adding 'data_juicer/analysis/diversity_analysis.py' 2024-12-20T13:13:51,783 INFO:wheel:adding 'data_juicer/analysis/draw.py' 2024-12-20T13:13:51,785 INFO:wheel:adding 'data_juicer/analysis/measure.py' 2024-12-20T13:13:51,786 INFO:wheel:adding 'data_juicer/analysis/overall_analysis.py' 2024-12-20T13:13:51,788 INFO:wheel:adding 'data_juicer/config/__init__.py' 2024-12-20T13:13:51,792 INFO:wheel:adding 'data_juicer/config/config.py' 2024-12-20T13:13:51,794 INFO:wheel:adding 'data_juicer/core/__init__.py' 2024-12-20T13:13:51,796 INFO:wheel:adding 'data_juicer/core/adapter.py' 2024-12-20T13:13:51,798 INFO:wheel:adding 'data_juicer/core/analyzer.py' 2024-12-20T13:13:51,800 INFO:wheel:adding 'data_juicer/core/data.py' 2024-12-20T13:13:51,802 INFO:wheel:adding 'data_juicer/core/executor.py' 2024-12-20T13:13:51,804 INFO:wheel:adding 'data_juicer/core/exporter.py' 2024-12-20T13:13:51,805 INFO:wheel:adding 'data_juicer/core/monitor.py' 2024-12-20T13:13:51,807 INFO:wheel:adding 'data_juicer/core/ray_data.py' 2024-12-20T13:13:51,808 INFO:wheel:adding 'data_juicer/core/ray_executor.py' 2024-12-20T13:13:51,810 INFO:wheel:adding 'data_juicer/core/tracer.py' 2024-12-20T13:13:51,812 INFO:wheel:adding 'data_juicer/format/__init__.py' 2024-12-20T13:13:51,813 INFO:wheel:adding 'data_juicer/format/csv_formatter.py' 2024-12-20T13:13:51,814 INFO:wheel:adding 'data_juicer/format/empty_formatter.py' 2024-12-20T13:13:51,816 INFO:wheel:adding 'data_juicer/format/formatter.py' 2024-12-20T13:13:51,817 INFO:wheel:adding 'data_juicer/format/json_formatter.py' 2024-12-20T13:13:51,818 INFO:wheel:adding 'data_juicer/format/load.py' 2024-12-20T13:13:51,820 INFO:wheel:adding 'data_juicer/format/mixture_formatter.py' 2024-12-20T13:13:51,821 INFO:wheel:adding 'data_juicer/format/parquet_formatter.py' 2024-12-20T13:13:51,823 INFO:wheel:adding 'data_juicer/format/text_formatter.py' 2024-12-20T13:13:51,824 INFO:wheel:adding 'data_juicer/format/tsv_formatter.py' 2024-12-20T13:13:51,826 INFO:wheel:adding 'data_juicer/ops/__init__.py' 2024-12-20T13:13:51,828 INFO:wheel:adding 'data_juicer/ops/base_op.py' 2024-12-20T13:13:51,829 INFO:wheel:adding 'data_juicer/ops/load.py' 2024-12-20T13:13:51,831 INFO:wheel:adding 'data_juicer/ops/op_fusion.py' 2024-12-20T13:13:51,833 INFO:wheel:adding 'data_juicer/ops/aggregator/__init__.py' 2024-12-20T13:13:51,835 INFO:wheel:adding 'data_juicer/ops/aggregator/entity_attribute_aggregator.py' 2024-12-20T13:13:51,836 INFO:wheel:adding 'data_juicer/ops/aggregator/most_relavant_entities_aggregator.py' 2024-12-20T13:13:51,838 INFO:wheel:adding 'data_juicer/ops/aggregator/nested_aggregator.py' 2024-12-20T13:13:51,840 INFO:wheel:adding 'data_juicer/ops/common/__init__.py' 2024-12-20T13:13:51,842 INFO:wheel:adding 'data_juicer/ops/common/helper_func.py' 2024-12-20T13:13:51,843 INFO:wheel:adding 'data_juicer/ops/common/special_characters.py' 2024-12-20T13:13:51,845 INFO:wheel:adding 'data_juicer/ops/deduplicator/__init__.py' 2024-12-20T13:13:51,846 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_deduplicator.py' 2024-12-20T13:13:51,848 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_minhash_deduplicator.py' 2024-12-20T13:13:51,850 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_simhash_deduplicator.py' 2024-12-20T13:13:51,852 INFO:wheel:adding 'data_juicer/ops/deduplicator/image_deduplicator.py' 2024-12-20T13:13:51,853 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_basic_deduplicator.py' 2024-12-20T13:13:51,855 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_document_deduplicator.py' 2024-12-20T13:13:51,856 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_image_deduplicator.py' 2024-12-20T13:13:51,857 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_video_deduplicator.py' 2024-12-20T13:13:51,859 INFO:wheel:adding 'data_juicer/ops/deduplicator/video_deduplicator.py' 2024-12-20T13:13:51,861 INFO:wheel:adding 'data_juicer/ops/filter/__init__.py' 2024-12-20T13:13:51,862 INFO:wheel:adding 'data_juicer/ops/filter/alphanumeric_filter.py' 2024-12-20T13:13:51,863 INFO:wheel:adding 'data_juicer/ops/filter/audio_duration_filter.py' 2024-12-20T13:13:51,865 INFO:wheel:adding 'data_juicer/ops/filter/audio_nmf_snr_filter.py' 2024-12-20T13:13:51,866 INFO:wheel:adding 'data_juicer/ops/filter/audio_size_filter.py' 2024-12-20T13:13:51,867 INFO:wheel:adding 'data_juicer/ops/filter/average_line_length_filter.py' 2024-12-20T13:13:51,868 INFO:wheel:adding 'data_juicer/ops/filter/character_repetition_filter.py' 2024-12-20T13:13:51,870 INFO:wheel:adding 'data_juicer/ops/filter/flagged_words_filter.py' 2024-12-20T13:13:51,871 INFO:wheel:adding 'data_juicer/ops/filter/image_aesthetics_filter.py' 2024-12-20T13:13:51,872 INFO:wheel:adding 'data_juicer/ops/filter/image_aspect_ratio_filter.py' 2024-12-20T13:13:51,874 INFO:wheel:adding 'data_juicer/ops/filter/image_face_count_filter.py' 2024-12-20T13:13:51,875 INFO:wheel:adding 'data_juicer/ops/filter/image_face_ratio_filter.py' 2024-12-20T13:13:51,877 INFO:wheel:adding 'data_juicer/ops/filter/image_nsfw_filter.py' 2024-12-20T13:13:51,878 INFO:wheel:adding 'data_juicer/ops/filter/image_pair_similarity_filter.py' 2024-12-20T13:13:51,879 INFO:wheel:adding 'data_juicer/ops/filter/image_shape_filter.py' 2024-12-20T13:13:51,880 INFO:wheel:adding 'data_juicer/ops/filter/image_size_filter.py' 2024-12-20T13:13:51,882 INFO:wheel:adding 'data_juicer/ops/filter/image_text_matching_filter.py' 2024-12-20T13:13:51,883 INFO:wheel:adding 'data_juicer/ops/filter/image_text_similarity_filter.py' 2024-12-20T13:13:51,885 INFO:wheel:adding 'data_juicer/ops/filter/image_watermark_filter.py' 2024-12-20T13:13:51,886 INFO:wheel:adding 'data_juicer/ops/filter/language_id_score_filter.py' 2024-12-20T13:13:51,887 INFO:wheel:adding 'data_juicer/ops/filter/maximum_line_length_filter.py' 2024-12-20T13:13:51,889 INFO:wheel:adding 'data_juicer/ops/filter/perplexity_filter.py' 2024-12-20T13:13:51,890 INFO:wheel:adding 'data_juicer/ops/filter/phrase_grounding_recall_filter.py' 2024-12-20T13:13:51,892 INFO:wheel:adding 'data_juicer/ops/filter/special_characters_filter.py' 2024-12-20T13:13:51,893 INFO:wheel:adding 'data_juicer/ops/filter/specified_field_filter.py' 2024-12-20T13:13:51,895 INFO:wheel:adding 'data_juicer/ops/filter/specified_numeric_field_filter.py' 2024-12-20T13:13:51,896 INFO:wheel:adding 'data_juicer/ops/filter/stopwords_filter.py' 2024-12-20T13:13:51,897 INFO:wheel:adding 'data_juicer/ops/filter/suffix_filter.py' 2024-12-20T13:13:51,899 INFO:wheel:adding 'data_juicer/ops/filter/text_action_filter.py' 2024-12-20T13:13:51,900 INFO:wheel:adding 'data_juicer/ops/filter/text_entity_dependency_filter.py' 2024-12-20T13:13:51,901 INFO:wheel:adding 'data_juicer/ops/filter/text_length_filter.py' 2024-12-20T13:13:51,903 INFO:wheel:adding 'data_juicer/ops/filter/token_num_filter.py' 2024-12-20T13:13:51,904 INFO:wheel:adding 'data_juicer/ops/filter/video_aesthetics_filter.py' 2024-12-20T13:13:51,906 INFO:wheel:adding 'data_juicer/ops/filter/video_aspect_ratio_filter.py' 2024-12-20T13:13:51,907 INFO:wheel:adding 'data_juicer/ops/filter/video_duration_filter.py' 2024-12-20T13:13:51,909 INFO:wheel:adding 'data_juicer/ops/filter/video_frames_text_similarity_filter.py' 2024-12-20T13:13:51,910 INFO:wheel:adding 'data_juicer/ops/filter/video_motion_score_filter.py' 2024-12-20T13:13:51,912 INFO:wheel:adding 'data_juicer/ops/filter/video_motion_score_raft_filter.py' 2024-12-20T13:13:51,914 INFO:wheel:adding 'data_juicer/ops/filter/video_nsfw_filter.py' 2024-12-20T13:13:51,915 INFO:wheel:adding 'data_juicer/ops/filter/video_ocr_area_ratio_filter.py' 2024-12-20T13:13:51,917 INFO:wheel:adding 'data_juicer/ops/filter/video_resolution_filter.py' 2024-12-20T13:13:51,918 INFO:wheel:adding 'data_juicer/ops/filter/video_tagging_from_frames_filter.py' 2024-12-20T13:13:51,920 INFO:wheel:adding 'data_juicer/ops/filter/video_watermark_filter.py' 2024-12-20T13:13:51,921 INFO:wheel:adding 'data_juicer/ops/filter/word_repetition_filter.py' 2024-12-20T13:13:51,922 INFO:wheel:adding 'data_juicer/ops/filter/words_num_filter.py' 2024-12-20T13:13:51,924 INFO:wheel:adding 'data_juicer/ops/grouper/__init__.py' 2024-12-20T13:13:51,925 INFO:wheel:adding 'data_juicer/ops/grouper/key_value_grouper.py' 2024-12-20T13:13:51,926 INFO:wheel:adding 'data_juicer/ops/grouper/naive_grouper.py' 2024-12-20T13:13:51,929 INFO:wheel:adding 'data_juicer/ops/mapper/__init__.py' 2024-12-20T13:13:51,930 INFO:wheel:adding 'data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py' 2024-12-20T13:13:51,932 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_qa_mapper.py' 2024-12-20T13:13:51,933 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_query_mapper.py' 2024-12-20T13:13:51,934 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_response_mapper.py' 2024-12-20T13:13:51,936 INFO:wheel:adding 'data_juicer/ops/mapper/chinese_convert_mapper.py' 2024-12-20T13:13:51,950 INFO:wheel:adding 'data_juicer/ops/mapper/clean_copyright_mapper.py' 2024-12-20T13:13:51,953 INFO:wheel:adding 'data_juicer/ops/mapper/clean_email_mapper.py' 2024-12-20T13:13:51,955 INFO:wheel:adding 'data_juicer/ops/mapper/clean_html_mapper.py' 2024-12-20T13:13:51,956 INFO:wheel:adding 'data_juicer/ops/mapper/clean_ip_mapper.py' 2024-12-20T13:13:51,957 INFO:wheel:adding 'data_juicer/ops/mapper/clean_links_mapper.py' 2024-12-20T13:13:51,958 INFO:wheel:adding 'data_juicer/ops/mapper/expand_macro_mapper.py' 2024-12-20T13:13:51,960 INFO:wheel:adding 'data_juicer/ops/mapper/extract_entity_attribute_mapper.py' 2024-12-20T13:13:51,963 INFO:wheel:adding 'data_juicer/ops/mapper/extract_entity_relation_mapper.py' 2024-12-20T13:13:51,964 INFO:wheel:adding 'data_juicer/ops/mapper/extract_event_mapper.py' 2024-12-20T13:13:51,966 INFO:wheel:adding 'data_juicer/ops/mapper/extract_keyword_mapper.py' 2024-12-20T13:13:51,967 INFO:wheel:adding 'data_juicer/ops/mapper/extract_nickname_mapper.py' 2024-12-20T13:13:51,969 INFO:wheel:adding 'data_juicer/ops/mapper/extract_support_text_mapper.py' 2024-12-20T13:13:51,970 INFO:wheel:adding 'data_juicer/ops/mapper/fix_unicode_mapper.py' 2024-12-20T13:13:51,972 INFO:wheel:adding 'data_juicer/ops/mapper/generate_qa_from_examples_mapper.py' 2024-12-20T13:13:51,974 INFO:wheel:adding 'data_juicer/ops/mapper/generate_qa_from_text_mapper.py' 2024-12-20T13:13:51,975 INFO:wheel:adding 'data_juicer/ops/mapper/image_blur_mapper.py' 2024-12-20T13:13:51,977 INFO:wheel:adding 'data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py' 2024-12-20T13:13:51,979 INFO:wheel:adding 'data_juicer/ops/mapper/image_captioning_mapper.py' 2024-12-20T13:13:51,981 INFO:wheel:adding 'data_juicer/ops/mapper/image_diffusion_mapper.py' 2024-12-20T13:13:51,983 INFO:wheel:adding 'data_juicer/ops/mapper/image_face_blur_mapper.py' 2024-12-20T13:13:51,984 INFO:wheel:adding 'data_juicer/ops/mapper/image_tagging_mapper.py' 2024-12-20T13:13:51,986 INFO:wheel:adding 'data_juicer/ops/mapper/nlpaug_en_mapper.py' 2024-12-20T13:13:51,988 INFO:wheel:adding 'data_juicer/ops/mapper/nlpcda_zh_mapper.py' 2024-12-20T13:13:51,989 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_qa_mapper.py' 2024-12-20T13:13:51,990 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_query_mapper.py' 2024-12-20T13:13:51,992 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_response_mapper.py' 2024-12-20T13:13:51,993 INFO:wheel:adding 'data_juicer/ops/mapper/pair_preference_mapper.py' 2024-12-20T13:13:51,995 INFO:wheel:adding 'data_juicer/ops/mapper/punctuation_normalization_mapper.py' 2024-12-20T13:13:51,996 INFO:wheel:adding 'data_juicer/ops/mapper/python_file_mapper.py' 2024-12-20T13:13:51,997 INFO:wheel:adding 'data_juicer/ops/mapper/python_lambda_mapper.py' 2024-12-20T13:13:51,999 INFO:wheel:adding 'data_juicer/ops/mapper/relation_identity_mapper.py' 2024-12-20T13:13:52,000 INFO:wheel:adding 'data_juicer/ops/mapper/remove_bibliography_mapper.py' 2024-12-20T13:13:52,001 INFO:wheel:adding 'data_juicer/ops/mapper/remove_comments_mapper.py' 2024-12-20T13:13:52,003 INFO:wheel:adding 'data_juicer/ops/mapper/remove_header_mapper.py' 2024-12-20T13:13:52,004 INFO:wheel:adding 'data_juicer/ops/mapper/remove_long_words_mapper.py' 2024-12-20T13:13:52,005 INFO:wheel:adding 'data_juicer/ops/mapper/remove_non_chinese_character_mapper.py' 2024-12-20T13:13:52,006 INFO:wheel:adding 'data_juicer/ops/mapper/remove_repeat_sentences_mapper.py' 2024-12-20T13:13:52,008 INFO:wheel:adding 'data_juicer/ops/mapper/remove_specific_chars_mapper.py' 2024-12-20T13:13:52,009 INFO:wheel:adding 'data_juicer/ops/mapper/remove_table_text_mapper.py' 2024-12-20T13:13:52,010 INFO:wheel:adding 'data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py' 2024-12-20T13:13:52,011 INFO:wheel:adding 'data_juicer/ops/mapper/replace_content_mapper.py' 2024-12-20T13:13:52,013 INFO:wheel:adding 'data_juicer/ops/mapper/sentence_split_mapper.py' 2024-12-20T13:13:52,014 INFO:wheel:adding 'data_juicer/ops/mapper/text_chunk_mapper.py' 2024-12-20T13:13:52,016 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_audio_mapper.py' 2024-12-20T13:13:52,018 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_frames_mapper.py' 2024-12-20T13:13:52,020 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py' 2024-12-20T13:13:52,022 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_video_mapper.py' 2024-12-20T13:13:52,024 INFO:wheel:adding 'data_juicer/ops/mapper/video_extract_frames_mapper.py' 2024-12-20T13:13:52,025 INFO:wheel:adding 'data_juicer/ops/mapper/video_face_blur_mapper.py' 2024-12-20T13:13:52,027 INFO:wheel:adding 'data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py' 2024-12-20T13:13:52,029 INFO:wheel:adding 'data_juicer/ops/mapper/video_remove_watermark_mapper.py' 2024-12-20T13:13:52,030 INFO:wheel:adding 'data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py' 2024-12-20T13:13:52,032 INFO:wheel:adding 'data_juicer/ops/mapper/video_resize_resolution_mapper.py' 2024-12-20T13:13:52,034 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_duration_mapper.py' 2024-12-20T13:13:52,035 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_key_frame_mapper.py' 2024-12-20T13:13:52,037 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_scene_mapper.py' 2024-12-20T13:13:52,038 INFO:wheel:adding 'data_juicer/ops/mapper/video_tagging_from_audio_mapper.py' 2024-12-20T13:13:52,040 INFO:wheel:adding 'data_juicer/ops/mapper/video_tagging_from_frames_mapper.py' 2024-12-20T13:13:52,041 INFO:wheel:adding 'data_juicer/ops/mapper/whitespace_normalization_mapper.py' 2024-12-20T13:13:52,043 INFO:wheel:adding 'data_juicer/ops/selector/__init__.py' 2024-12-20T13:13:52,044 INFO:wheel:adding 'data_juicer/ops/selector/frequency_specified_field_selector.py' 2024-12-20T13:13:52,045 INFO:wheel:adding 'data_juicer/ops/selector/random_selector.py' 2024-12-20T13:13:52,047 INFO:wheel:adding 'data_juicer/ops/selector/range_specified_field_selector.py' 2024-12-20T13:13:52,048 INFO:wheel:adding 'data_juicer/ops/selector/topk_specified_field_selector.py' 2024-12-20T13:13:52,050 INFO:wheel:adding 'data_juicer/tools/__init__.py' 2024-12-20T13:13:52,051 INFO:wheel:adding 'data_juicer/tools/analyze_data.py' 2024-12-20T13:13:52,052 INFO:wheel:adding 'data_juicer/tools/dj_install.py' 2024-12-20T13:13:52,053 INFO:wheel:adding 'data_juicer/tools/process_data.py' 2024-12-20T13:13:52,055 INFO:wheel:adding 'data_juicer/tools/sandbox_starter.py' 2024-12-20T13:13:52,057 INFO:wheel:adding 'data_juicer/utils/__init__.py' 2024-12-20T13:13:52,058 INFO:wheel:adding 'data_juicer/utils/asset_utils.py' 2024-12-20T13:13:52,059 INFO:wheel:adding 'data_juicer/utils/auto_install_mapping.py' 2024-12-20T13:13:52,061 INFO:wheel:adding 'data_juicer/utils/auto_install_utils.py' 2024-12-20T13:13:52,062 INFO:wheel:adding 'data_juicer/utils/availability_utils.py' 2024-12-20T13:13:52,063 INFO:wheel:adding 'data_juicer/utils/cache_utils.py' 2024-12-20T13:13:52,064 INFO:wheel:adding 'data_juicer/utils/ckpt_utils.py' 2024-12-20T13:13:52,066 INFO:wheel:adding 'data_juicer/utils/common_utils.py' 2024-12-20T13:13:52,068 INFO:wheel:adding 'data_juicer/utils/compress.py' 2024-12-20T13:13:52,070 INFO:wheel:adding 'data_juicer/utils/constant.py' 2024-12-20T13:13:52,071 INFO:wheel:adding 'data_juicer/utils/file_utils.py' 2024-12-20T13:13:52,073 INFO:wheel:adding 'data_juicer/utils/fingerprint_utils.py' 2024-12-20T13:13:52,074 INFO:wheel:adding 'data_juicer/utils/lazy_loader.py' 2024-12-20T13:13:52,075 INFO:wheel:adding 'data_juicer/utils/logger_utils.py' 2024-12-20T13:13:52,079 INFO:wheel:adding 'data_juicer/utils/mm_utils.py' 2024-12-20T13:13:52,083 INFO:wheel:adding 'data_juicer/utils/model_utils.py' 2024-12-20T13:13:52,085 INFO:wheel:adding 'data_juicer/utils/process_utils.py' 2024-12-20T13:13:52,087 INFO:wheel:adding 'data_juicer/utils/registry.py' 2024-12-20T13:13:52,088 INFO:wheel:adding 'data_juicer/utils/resource_utils.py' 2024-12-20T13:13:52,089 INFO:wheel:adding 'data_juicer/utils/unittest_utils.py' 2024-12-20T13:13:52,093 INFO:wheel:adding 'py_data_juicer-1.0.2.dist-info/LICENSE' 2024-12-20T13:13:52,097 INFO:wheel:adding 'py_data_juicer-1.0.2.dist-info/METADATA' 2024-12-20T13:13:52,098 INFO:wheel:adding 'py_data_juicer-1.0.2.dist-info/WHEEL' 2024-12-20T13:13:52,099 INFO:wheel:adding 'py_data_juicer-1.0.2.dist-info/entry_points.txt' 2024-12-20T13:13:52,100 INFO:wheel:adding 'py_data_juicer-1.0.2.dist-info/top_level.txt' 2024-12-20T13:13:52,103 INFO:wheel:adding 'py_data_juicer-1.0.2.dist-info/RECORD' 2024-12-20T13:13:52,110 INFO:root:removing build/bdist.linux-armv7l/wheel 2024-12-20T13:13:52,247 Building wheel for py-data-juicer (setup.py): finished with status 'done' 2024-12-20T13:13:52,253 Created wheel for py-data-juicer: filename=py_data_juicer-1.0.2-py3-none-any.whl size=358947 sha256=69bc91a28885bd8df218510cd9c0198397f76eaab0c7fbf9726181da70734781 2024-12-20T13:13:52,255 Stored in directory: /tmp/pip-ephem-wheel-cache-pzdykz2i/wheels/95/56/3e/70431becc91eeec95919597dc3243cd93bbbc0466ef710f1c7 2024-12-20T13:13:52,276 Successfully built py-data-juicer 2024-12-20T13:13:52,291 Removed build tracker: '/tmp/pip-build-tracker-u3keqqh8'