2025-01-03T11:40:56,535 Created temporary directory: /tmp/pip-build-tracker-gufl43z0 2025-01-03T11:40:56,536 Initialized build tracking at /tmp/pip-build-tracker-gufl43z0 2025-01-03T11:40:56,536 Created build tracker: /tmp/pip-build-tracker-gufl43z0 2025-01-03T11:40:56,537 Entered build tracker: /tmp/pip-build-tracker-gufl43z0 2025-01-03T11:40:56,538 Created temporary directory: /tmp/pip-wheel-gc8ow8rq 2025-01-03T11:40:56,542 Created temporary directory: /tmp/pip-ephem-wheel-cache-_r962h76 2025-01-03T11:40:56,585 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2025-01-03T11:40:56,588 2 location(s) to search for versions of py-data-juicer: 2025-01-03T11:40:56,588 * https://pypi.org/simple/py-data-juicer/ 2025-01-03T11:40:56,588 * https://www.piwheels.org/simple/py-data-juicer/ 2025-01-03T11:40:56,589 Fetching project page and analyzing links: https://pypi.org/simple/py-data-juicer/ 2025-01-03T11:40:56,590 Getting page https://pypi.org/simple/py-data-juicer/ 2025-01-03T11:40:56,591 Found index url https://pypi.org/simple/ 2025-01-03T11:40:56,811 Fetched page https://pypi.org/simple/py-data-juicer/ as application/vnd.pypi.simple.v1+json 2025-01-03T11:40:56,815 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/e3/d3/d0724d922e7c55a0664485fdf642124bdcd801df2697e29c9463c4360958/py_data_juicer-0.1.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-03T11:40:56,816 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/fd/c6/e1428310bf534319ddc2e362c6eee21985f097e70cedc0e9cd3e914de826/py_data_juicer-0.1.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-03T11:40:56,816 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/67/70/731e349d2a92bf59a767230b956665dd27ea24a7a948d4a1710154d77a24/py_data_juicer-0.1.2-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-03T11:40:56,817 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/66/08/f584efbdf8277a061ce73c9befc33fec813cbd08020761af2380dd331692/py_data_juicer-0.1.3-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-03T11:40:56,818 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/69/20/9862dfe7a94f10caa0e6387834d1265420613345c7adadd41a3ced7230e8/py_data_juicer-0.2.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-03T11:40:56,818 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/05/34/33fb401d350f1a66cdaefc224419f9d01484722829f5b0c71ef59a8365a4/py_data_juicer-1.0.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-03T11:40:56,819 Found link https://files.pythonhosted.org/packages/49/e9/cfb994255490c36554048e0b3956858e61f5a75ffacfde20849f31f04ef8/py_data_juicer-1.0.0.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.0 2025-01-03T11:40:56,819 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/27/37/150e2198f14349fdd6b647a63feaede809e21dbfc4617aba487f18049828/py_data_juicer-1.0.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-03T11:40:56,820 Found link https://files.pythonhosted.org/packages/4a/60/dadcbe4337a76d8f98022b60d142ac31073d568c9e06a5461b3014d16fc6/py_data_juicer-1.0.1.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.1 2025-01-03T11:40:56,821 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/10/dd/f9cadd6ed2f19c4f94e61a3fd4bb5bb2029c20b2cf61ec86c3a05f89f74b/py_data_juicer-1.0.2-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-03T11:40:56,822 Found link https://files.pythonhosted.org/packages/95/8f/c0447ce091ca0b86283e8074a3cb350572a5436ca13dceb288887a0de332/py_data_juicer-1.0.2.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.2 2025-01-03T11:40:56,822 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/1f/dd/0f1804c5cfce50c52c8b60a1b710f30d893f681e42a6acc41bfd29a20059/py_data_juicer-1.0.3-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-03T11:40:56,823 Found link https://files.pythonhosted.org/packages/6e/d0/eefb1ca00cd4c8e62c0e8f62a1b5357b961ce50c5d2647107f51f68bf128/py_data_juicer-1.0.3.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.3 2025-01-03T11:40:56,824 Fetching project page and analyzing links: https://www.piwheels.org/simple/py-data-juicer/ 2025-01-03T11:40:56,824 Getting page https://www.piwheels.org/simple/py-data-juicer/ 2025-01-03T11:40:56,826 Found index url https://www.piwheels.org/simple/ 2025-01-03T11:40:56,989 Fetched page https://www.piwheels.org/simple/py-data-juicer/ as text/html 2025-01-03T11:40:56,991 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.2-py3-none-any.whl#sha256=69bc91a28885bd8df218510cd9c0198397f76eaab0c7fbf9726181da70734781 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-01-03T11:40:56,992 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.1-py3-none-any.whl#sha256=2396c39923ab13a9fe17fe6523734f2ff41b3edb2c97fcc70539a5761e0465f8 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-01-03T11:40:56,993 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.0-py3-none-any.whl#sha256=7cd007456933447474ec9faf5357b741a63e7a0c0dc6230e40bf4498eb1ef554 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-01-03T11:40:56,993 Skipping link: not a file: https://www.piwheels.org/simple/py-data-juicer/ 2025-01-03T11:40:56,994 Skipping link: not a file: https://pypi.org/simple/py-data-juicer/ 2025-01-03T11:40:57,012 Given no hashes to check 1 links for project 'py-data-juicer': discarding no candidates 2025-01-03T11:40:57,014 Collecting py-data-juicer==1.0.3 2025-01-03T11:40:57,016 Created temporary directory: /tmp/pip-unpack-g42papri 2025-01-03T11:40:57,233 Downloading py_data_juicer-1.0.3.tar.gz (262 kB) 2025-01-03T11:40:57,750 Added py-data-juicer==1.0.3 from https://files.pythonhosted.org/packages/6e/d0/eefb1ca00cd4c8e62c0e8f62a1b5357b961ce50c5d2647107f51f68bf128/py_data_juicer-1.0.3.tar.gz to build tracker '/tmp/pip-build-tracker-gufl43z0' 2025-01-03T11:40:57,752 Running setup.py (path:/tmp/pip-wheel-gc8ow8rq/py-data-juicer_0711fb677b7745a1a1d7f132504c1733/setup.py) egg_info for package py-data-juicer 2025-01-03T11:40:57,753 Created temporary directory: /tmp/pip-pip-egg-info-scupuxml 2025-01-03T11:40:57,753 Preparing metadata (setup.py): started 2025-01-03T11:40:57,755 Running command python setup.py egg_info 2025-01-03T11:40:58,252 WARNING:root:target file does not exist: environments/minimal_requires.txt 2025-01-03T11:40:58,252 WARNING:root:target file does not exist: environments/science_requires.txt 2025-01-03T11:40:58,253 WARNING:root:target file does not exist: environments/dist_requires.txt 2025-01-03T11:40:58,254 WARNING:root:target file does not exist: environments/dev_requires.txt 2025-01-03T11:40:58,254 WARNING:root:target file does not exist: environments/preprocess_requires.txt 2025-01-03T11:40:58,255 WARNING:root:target file does not exist: environments/quality_classifier_requires.txt 2025-01-03T11:40:58,255 WARNING:root:target file does not exist: environments/sandbox_requires.txt 2025-01-03T11:40:58,879 INFO:root:running egg_info 2025-01-03T11:40:58,909 INFO:root:creating /tmp/pip-pip-egg-info-scupuxml/py_data_juicer.egg-info 2025-01-03T11:40:58,910 INFO:root:writing /tmp/pip-pip-egg-info-scupuxml/py_data_juicer.egg-info/PKG-INFO 2025-01-03T11:40:58,914 INFO:root:writing dependency_links to /tmp/pip-pip-egg-info-scupuxml/py_data_juicer.egg-info/dependency_links.txt 2025-01-03T11:40:58,916 INFO:root:writing entry points to /tmp/pip-pip-egg-info-scupuxml/py_data_juicer.egg-info/entry_points.txt 2025-01-03T11:40:58,918 INFO:root:writing requirements to /tmp/pip-pip-egg-info-scupuxml/py_data_juicer.egg-info/requires.txt 2025-01-03T11:40:58,919 INFO:root:writing top-level names to /tmp/pip-pip-egg-info-scupuxml/py_data_juicer.egg-info/top_level.txt 2025-01-03T11:40:58,920 INFO:root:writing manifest file '/tmp/pip-pip-egg-info-scupuxml/py_data_juicer.egg-info/SOURCES.txt' 2025-01-03T11:40:59,034 INFO:root:reading manifest file '/tmp/pip-pip-egg-info-scupuxml/py_data_juicer.egg-info/SOURCES.txt' 2025-01-03T11:40:59,036 INFO:root:adding license file 'LICENSE' 2025-01-03T11:40:59,045 INFO:root:writing manifest file '/tmp/pip-pip-egg-info-scupuxml/py_data_juicer.egg-info/SOURCES.txt' 2025-01-03T11:40:59,136 Preparing metadata (setup.py): finished with status 'done' 2025-01-03T11:40:59,143 Source in /tmp/pip-wheel-gc8ow8rq/py-data-juicer_0711fb677b7745a1a1d7f132504c1733 has version 1.0.3, which satisfies requirement py-data-juicer==1.0.3 from https://files.pythonhosted.org/packages/6e/d0/eefb1ca00cd4c8e62c0e8f62a1b5357b961ce50c5d2647107f51f68bf128/py_data_juicer-1.0.3.tar.gz 2025-01-03T11:40:59,145 Removed py-data-juicer==1.0.3 from https://files.pythonhosted.org/packages/6e/d0/eefb1ca00cd4c8e62c0e8f62a1b5357b961ce50c5d2647107f51f68bf128/py_data_juicer-1.0.3.tar.gz from build tracker '/tmp/pip-build-tracker-gufl43z0' 2025-01-03T11:40:59,153 Created temporary directory: /tmp/pip-unpack-5utki1_l 2025-01-03T11:40:59,154 Created temporary directory: /tmp/pip-unpack-9_4f6up5 2025-01-03T11:40:59,154 Building wheels for collected packages: py-data-juicer 2025-01-03T11:40:59,161 Created temporary directory: /tmp/pip-wheel-1ew9idva 2025-01-03T11:40:59,161 Building wheel for py-data-juicer (setup.py): started 2025-01-03T11:40:59,163 Destination directory: /tmp/pip-wheel-1ew9idva 2025-01-03T11:40:59,163 Running command python setup.py bdist_wheel 2025-01-03T11:40:59,630 WARNING:root:target file does not exist: environments/minimal_requires.txt 2025-01-03T11:40:59,630 WARNING:root:target file does not exist: environments/science_requires.txt 2025-01-03T11:40:59,631 WARNING:root:target file does not exist: environments/dist_requires.txt 2025-01-03T11:40:59,631 WARNING:root:target file does not exist: environments/dev_requires.txt 2025-01-03T11:40:59,632 WARNING:root:target file does not exist: environments/preprocess_requires.txt 2025-01-03T11:40:59,633 WARNING:root:target file does not exist: environments/quality_classifier_requires.txt 2025-01-03T11:40:59,633 WARNING:root:target file does not exist: environments/sandbox_requires.txt 2025-01-03T11:41:00,224 INFO:root:running bdist_wheel 2025-01-03T11:41:00,365 INFO:root:running build 2025-01-03T11:41:00,366 INFO:root:running build_py 2025-01-03T11:41:00,397 INFO:root:creating build/lib/data_juicer 2025-01-03T11:41:00,399 INFO:root:copying data_juicer/__init__.py -> build/lib/data_juicer 2025-01-03T11:41:00,402 INFO:root:creating build/lib/data_juicer/format 2025-01-03T11:41:00,404 INFO:root:copying data_juicer/format/formatter.py -> build/lib/data_juicer/format 2025-01-03T11:41:00,406 INFO:root:copying data_juicer/format/mixture_formatter.py -> build/lib/data_juicer/format 2025-01-03T11:41:00,409 INFO:root:copying data_juicer/format/text_formatter.py -> build/lib/data_juicer/format 2025-01-03T11:41:00,411 INFO:root:copying data_juicer/format/parquet_formatter.py -> build/lib/data_juicer/format 2025-01-03T11:41:00,413 INFO:root:copying data_juicer/format/empty_formatter.py -> build/lib/data_juicer/format 2025-01-03T11:41:00,415 INFO:root:copying data_juicer/format/load.py -> build/lib/data_juicer/format 2025-01-03T11:41:00,417 INFO:root:copying data_juicer/format/tsv_formatter.py -> build/lib/data_juicer/format 2025-01-03T11:41:00,419 INFO:root:copying data_juicer/format/csv_formatter.py -> build/lib/data_juicer/format 2025-01-03T11:41:00,420 INFO:root:copying data_juicer/format/__init__.py -> build/lib/data_juicer/format 2025-01-03T11:41:00,422 INFO:root:copying data_juicer/format/json_formatter.py -> build/lib/data_juicer/format 2025-01-03T11:41:00,425 INFO:root:creating build/lib/data_juicer/analysis 2025-01-03T11:41:00,426 INFO:root:copying data_juicer/analysis/collector.py -> build/lib/data_juicer/analysis 2025-01-03T11:41:00,428 INFO:root:copying data_juicer/analysis/measure.py -> build/lib/data_juicer/analysis 2025-01-03T11:41:00,430 INFO:root:copying data_juicer/analysis/draw.py -> build/lib/data_juicer/analysis 2025-01-03T11:41:00,432 INFO:root:copying data_juicer/analysis/column_wise_analysis.py -> build/lib/data_juicer/analysis 2025-01-03T11:41:00,434 INFO:root:copying data_juicer/analysis/diversity_analysis.py -> build/lib/data_juicer/analysis 2025-01-03T11:41:00,437 INFO:root:copying data_juicer/analysis/__init__.py -> build/lib/data_juicer/analysis 2025-01-03T11:41:00,438 INFO:root:copying data_juicer/analysis/overall_analysis.py -> build/lib/data_juicer/analysis 2025-01-03T11:41:00,441 INFO:root:creating build/lib/data_juicer/config 2025-01-03T11:41:00,442 INFO:root:copying data_juicer/config/config.py -> build/lib/data_juicer/config 2025-01-03T11:41:00,445 INFO:root:copying data_juicer/config/__init__.py -> build/lib/data_juicer/config 2025-01-03T11:41:00,447 INFO:root:creating build/lib/data_juicer/utils 2025-01-03T11:41:00,448 INFO:root:copying data_juicer/utils/compress.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,451 INFO:root:copying data_juicer/utils/logger_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,453 INFO:root:copying data_juicer/utils/model_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,456 INFO:root:copying data_juicer/utils/resource_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,458 INFO:root:copying data_juicer/utils/auto_install_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,460 INFO:root:copying data_juicer/utils/process_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,462 INFO:root:copying data_juicer/utils/availability_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,464 INFO:root:copying data_juicer/utils/unittest_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,466 INFO:root:copying data_juicer/utils/asset_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,468 INFO:root:copying data_juicer/utils/cache_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,470 INFO:root:copying data_juicer/utils/mm_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,473 INFO:root:copying data_juicer/utils/constant.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,477 INFO:root:copying data_juicer/utils/auto_install_mapping.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,482 INFO:root:copying data_juicer/utils/file_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,484 INFO:root:copying data_juicer/utils/__init__.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,485 INFO:root:copying data_juicer/utils/lazy_loader.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,487 INFO:root:copying data_juicer/utils/common_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,489 INFO:root:copying data_juicer/utils/ckpt_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,491 INFO:root:copying data_juicer/utils/registry.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,494 INFO:root:copying data_juicer/utils/fingerprint_utils.py -> build/lib/data_juicer/utils 2025-01-03T11:41:00,500 INFO:root:creating build/lib/data_juicer/ops 2025-01-03T11:41:00,504 INFO:root:copying data_juicer/ops/base_op.py -> build/lib/data_juicer/ops 2025-01-03T11:41:00,507 INFO:root:copying data_juicer/ops/load.py -> build/lib/data_juicer/ops 2025-01-03T11:41:00,508 INFO:root:copying data_juicer/ops/__init__.py -> build/lib/data_juicer/ops 2025-01-03T11:41:00,512 INFO:root:copying data_juicer/ops/op_fusion.py -> build/lib/data_juicer/ops 2025-01-03T11:41:00,518 INFO:root:creating build/lib/data_juicer/core 2025-01-03T11:41:00,519 INFO:root:copying data_juicer/core/tracer.py -> build/lib/data_juicer/core 2025-01-03T11:41:00,522 INFO:root:copying data_juicer/core/exporter.py -> build/lib/data_juicer/core 2025-01-03T11:41:00,525 INFO:root:copying data_juicer/core/analyzer.py -> build/lib/data_juicer/core 2025-01-03T11:41:00,529 INFO:root:copying data_juicer/core/ray_executor.py -> build/lib/data_juicer/core 2025-01-03T11:41:00,531 INFO:root:copying data_juicer/core/data.py -> build/lib/data_juicer/core 2025-01-03T11:41:00,534 INFO:root:copying data_juicer/core/executor.py -> build/lib/data_juicer/core 2025-01-03T11:41:00,536 INFO:root:copying data_juicer/core/monitor.py -> build/lib/data_juicer/core 2025-01-03T11:41:00,539 INFO:root:copying data_juicer/core/__init__.py -> build/lib/data_juicer/core 2025-01-03T11:41:00,540 INFO:root:copying data_juicer/core/ray_data.py -> build/lib/data_juicer/core 2025-01-03T11:41:00,546 INFO:root:copying data_juicer/core/adapter.py -> build/lib/data_juicer/core 2025-01-03T11:41:00,557 INFO:root:creating build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,562 INFO:root:copying data_juicer/ops/deduplicator/video_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,569 INFO:root:copying data_juicer/ops/deduplicator/ray_basic_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,571 INFO:root:copying data_juicer/ops/deduplicator/ray_image_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,575 INFO:root:copying data_juicer/ops/deduplicator/document_minhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,580 INFO:root:copying data_juicer/ops/deduplicator/document_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,583 INFO:root:copying data_juicer/ops/deduplicator/__init__.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,586 INFO:root:copying data_juicer/ops/deduplicator/ray_bts_minhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,590 INFO:root:copying data_juicer/ops/deduplicator/document_simhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,595 INFO:root:copying data_juicer/ops/deduplicator/ray_video_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,597 INFO:root:copying data_juicer/ops/deduplicator/image_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,599 INFO:root:copying data_juicer/ops/deduplicator/ray_document_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-03T11:41:00,602 INFO:root:creating build/lib/data_juicer/ops/grouper 2025-01-03T11:41:00,603 INFO:root:copying data_juicer/ops/grouper/key_value_grouper.py -> build/lib/data_juicer/ops/grouper 2025-01-03T11:41:00,607 INFO:root:copying data_juicer/ops/grouper/naive_grouper.py -> build/lib/data_juicer/ops/grouper 2025-01-03T11:41:00,613 INFO:root:copying data_juicer/ops/grouper/naive_reverse_grouper.py -> build/lib/data_juicer/ops/grouper 2025-01-03T11:41:00,622 INFO:root:copying data_juicer/ops/grouper/__init__.py -> build/lib/data_juicer/ops/grouper 2025-01-03T11:41:00,631 INFO:root:creating build/lib/data_juicer/ops/selector 2025-01-03T11:41:00,632 INFO:root:copying data_juicer/ops/selector/topk_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-01-03T11:41:00,635 INFO:root:copying data_juicer/ops/selector/frequency_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-01-03T11:41:00,640 INFO:root:copying data_juicer/ops/selector/tags_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-01-03T11:41:00,648 INFO:root:copying data_juicer/ops/selector/random_selector.py -> build/lib/data_juicer/ops/selector 2025-01-03T11:41:00,652 INFO:root:copying data_juicer/ops/selector/range_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-01-03T11:41:00,655 INFO:root:copying data_juicer/ops/selector/__init__.py -> build/lib/data_juicer/ops/selector 2025-01-03T11:41:00,659 INFO:root:creating build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,660 INFO:root:copying data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,664 INFO:root:copying data_juicer/ops/mapper/generate_qa_from_examples_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,667 INFO:root:copying data_juicer/ops/mapper/punctuation_normalization_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,668 INFO:root:copying data_juicer/ops/mapper/image_diffusion_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,671 INFO:root:copying data_juicer/ops/mapper/video_resize_resolution_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,674 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,676 INFO:root:copying data_juicer/ops/mapper/remove_non_chinese_character_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,679 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,682 INFO:root:copying data_juicer/ops/mapper/image_face_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,685 INFO:root:copying data_juicer/ops/mapper/calibrate_response_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,688 INFO:root:copying data_juicer/ops/mapper/query_intent_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,690 INFO:root:copying data_juicer/ops/mapper/text_chunk_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,693 INFO:root:copying data_juicer/ops/mapper/remove_long_words_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,695 INFO:root:copying data_juicer/ops/mapper/expand_macro_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,697 INFO:root:copying data_juicer/ops/mapper/whitespace_normalization_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,700 INFO:root:copying data_juicer/ops/mapper/dialog_sentiment_intensity_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,703 INFO:root:copying data_juicer/ops/mapper/optimize_response_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,706 INFO:root:copying data_juicer/ops/mapper/extract_entity_attribute_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,708 INFO:root:copying data_juicer/ops/mapper/fix_unicode_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,710 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_video_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,713 INFO:root:copying data_juicer/ops/mapper/extract_event_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,715 INFO:root:copying data_juicer/ops/mapper/image_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,717 INFO:root:copying data_juicer/ops/mapper/clean_email_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,719 INFO:root:copying data_juicer/ops/mapper/pair_preference_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,721 INFO:root:copying data_juicer/ops/mapper/remove_repeat_sentences_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,723 INFO:root:copying data_juicer/ops/mapper/sentence_split_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,725 INFO:root:copying data_juicer/ops/mapper/query_sentiment_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,727 INFO:root:copying data_juicer/ops/mapper/python_lambda_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,729 INFO:root:copying data_juicer/ops/mapper/extract_entity_relation_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,732 INFO:root:copying data_juicer/ops/mapper/dialog_intent_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,734 INFO:root:copying data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,736 INFO:root:copying data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,738 INFO:root:copying data_juicer/ops/mapper/clean_links_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,740 INFO:root:copying data_juicer/ops/mapper/extract_support_text_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,742 INFO:root:copying data_juicer/ops/mapper/image_captioning_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,745 INFO:root:copying data_juicer/ops/mapper/image_tagging_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,747 INFO:root:copying data_juicer/ops/mapper/generate_qa_from_text_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,749 INFO:root:copying data_juicer/ops/mapper/query_topic_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,751 INFO:root:copying data_juicer/ops/mapper/video_split_by_scene_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,753 INFO:root:copying data_juicer/ops/mapper/remove_header_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,755 INFO:root:copying data_juicer/ops/mapper/chinese_convert_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,757 INFO:root:copying data_juicer/ops/mapper/video_extract_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,759 INFO:root:copying data_juicer/ops/mapper/extract_nickname_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,761 INFO:root:copying data_juicer/ops/mapper/optimize_qa_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,764 INFO:root:copying data_juicer/ops/mapper/video_tagging_from_audio_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,765 INFO:root:copying data_juicer/ops/mapper/clean_copyright_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,767 INFO:root:copying data_juicer/ops/mapper/nlpcda_zh_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,770 INFO:root:copying data_juicer/ops/mapper/clean_ip_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,772 INFO:root:copying data_juicer/ops/mapper/video_split_by_key_frame_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,774 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_audio_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,776 INFO:root:copying data_juicer/ops/mapper/clean_html_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,778 INFO:root:copying data_juicer/ops/mapper/remove_specific_chars_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,780 INFO:root:copying data_juicer/ops/mapper/remove_bibliography_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,782 INFO:root:copying data_juicer/ops/mapper/dialog_sentiment_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,784 INFO:root:copying data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,787 INFO:root:copying data_juicer/ops/mapper/video_tagging_from_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,789 INFO:root:copying data_juicer/ops/mapper/optimize_query_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,791 INFO:root:copying data_juicer/ops/mapper/__init__.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,793 INFO:root:copying data_juicer/ops/mapper/video_face_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,796 INFO:root:copying data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,797 INFO:root:copying data_juicer/ops/mapper/remove_table_text_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,799 INFO:root:copying data_juicer/ops/mapper/calibrate_query_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,801 INFO:root:copying data_juicer/ops/mapper/dialog_topic_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,804 INFO:root:copying data_juicer/ops/mapper/video_remove_watermark_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,806 INFO:root:copying data_juicer/ops/mapper/python_file_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,807 INFO:root:copying data_juicer/ops/mapper/replace_content_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,809 INFO:root:copying data_juicer/ops/mapper/extract_keyword_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,812 INFO:root:copying data_juicer/ops/mapper/calibrate_qa_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,814 INFO:root:copying data_juicer/ops/mapper/remove_comments_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,816 INFO:root:copying data_juicer/ops/mapper/video_split_by_duration_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,818 INFO:root:copying data_juicer/ops/mapper/relation_identity_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,821 INFO:root:copying data_juicer/ops/mapper/nlpaug_en_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-03T11:41:00,825 INFO:root:creating build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,827 INFO:root:copying data_juicer/ops/filter/video_watermark_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,834 INFO:root:copying data_juicer/ops/filter/video_motion_score_raft_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,837 INFO:root:copying data_juicer/ops/filter/image_nsfw_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,839 INFO:root:copying data_juicer/ops/filter/words_num_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,841 INFO:root:copying data_juicer/ops/filter/video_duration_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,843 INFO:root:copying data_juicer/ops/filter/flagged_words_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,845 INFO:root:copying data_juicer/ops/filter/perplexity_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,847 INFO:root:copying data_juicer/ops/filter/word_repetition_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,849 INFO:root:copying data_juicer/ops/filter/phrase_grounding_recall_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,852 INFO:root:copying data_juicer/ops/filter/specified_field_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,854 INFO:root:copying data_juicer/ops/filter/video_aspect_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,856 INFO:root:copying data_juicer/ops/filter/character_repetition_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,858 INFO:root:copying data_juicer/ops/filter/image_text_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,860 INFO:root:copying data_juicer/ops/filter/image_face_count_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,862 INFO:root:copying data_juicer/ops/filter/image_face_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,865 INFO:root:copying data_juicer/ops/filter/video_nsfw_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,867 INFO:root:copying data_juicer/ops/filter/maximum_line_length_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,869 INFO:root:copying data_juicer/ops/filter/stopwords_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,871 INFO:root:copying data_juicer/ops/filter/video_motion_score_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,873 INFO:root:copying data_juicer/ops/filter/text_length_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,875 INFO:root:copying data_juicer/ops/filter/image_aspect_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,877 INFO:root:copying data_juicer/ops/filter/audio_size_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,879 INFO:root:copying data_juicer/ops/filter/suffix_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,881 INFO:root:copying data_juicer/ops/filter/token_num_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,883 INFO:root:copying data_juicer/ops/filter/video_aesthetics_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,885 INFO:root:copying data_juicer/ops/filter/video_ocr_area_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,887 INFO:root:copying data_juicer/ops/filter/audio_duration_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,889 INFO:root:copying data_juicer/ops/filter/image_shape_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,891 INFO:root:copying data_juicer/ops/filter/average_line_length_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,892 INFO:root:copying data_juicer/ops/filter/image_size_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,894 INFO:root:copying data_juicer/ops/filter/image_aesthetics_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,896 INFO:root:copying data_juicer/ops/filter/image_watermark_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,899 INFO:root:copying data_juicer/ops/filter/video_resolution_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,901 INFO:root:copying data_juicer/ops/filter/language_id_score_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,903 INFO:root:copying data_juicer/ops/filter/__init__.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,904 INFO:root:copying data_juicer/ops/filter/text_entity_dependency_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,906 INFO:root:copying data_juicer/ops/filter/image_pair_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,908 INFO:root:copying data_juicer/ops/filter/text_action_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,910 INFO:root:copying data_juicer/ops/filter/image_text_matching_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,912 INFO:root:copying data_juicer/ops/filter/video_frames_text_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,915 INFO:root:copying data_juicer/ops/filter/special_characters_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,917 INFO:root:copying data_juicer/ops/filter/video_tagging_from_frames_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,919 INFO:root:copying data_juicer/ops/filter/specified_numeric_field_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,921 INFO:root:copying data_juicer/ops/filter/alphanumeric_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,923 INFO:root:copying data_juicer/ops/filter/audio_nmf_snr_filter.py -> build/lib/data_juicer/ops/filter 2025-01-03T11:41:00,926 INFO:root:creating build/lib/data_juicer/ops/common 2025-01-03T11:41:00,927 INFO:root:copying data_juicer/ops/common/special_characters.py -> build/lib/data_juicer/ops/common 2025-01-03T11:41:00,929 INFO:root:copying data_juicer/ops/common/helper_func.py -> build/lib/data_juicer/ops/common 2025-01-03T11:41:00,932 INFO:root:copying data_juicer/ops/common/__init__.py -> build/lib/data_juicer/ops/common 2025-01-03T11:41:00,934 INFO:root:creating build/lib/data_juicer/ops/aggregator 2025-01-03T11:41:00,935 INFO:root:copying data_juicer/ops/aggregator/entity_attribute_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-01-03T11:41:00,938 INFO:root:copying data_juicer/ops/aggregator/meta_tags_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-01-03T11:41:00,940 INFO:root:copying data_juicer/ops/aggregator/nested_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-01-03T11:41:00,943 INFO:root:copying data_juicer/ops/aggregator/__init__.py -> build/lib/data_juicer/ops/aggregator 2025-01-03T11:41:00,945 INFO:root:copying data_juicer/ops/aggregator/most_relavant_entities_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-01-03T11:41:00,948 INFO:root:creating build/lib/data_juicer/tools 2025-01-03T11:41:00,949 INFO:root:copying tools/sandbox_starter.py -> build/lib/data_juicer/tools 2025-01-03T11:41:00,950 INFO:root:copying tools/dj_install.py -> build/lib/data_juicer/tools 2025-01-03T11:41:00,952 INFO:root:copying tools/analyze_data.py -> build/lib/data_juicer/tools 2025-01-03T11:41:00,954 INFO:root:copying tools/__init__.py -> build/lib/data_juicer/tools 2025-01-03T11:41:00,955 INFO:root:copying tools/process_data.py -> build/lib/data_juicer/tools 2025-01-03T11:41:01,255 /usr/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. 2025-01-03T11:41:01,256 !! 2025-01-03T11:41:01,258 ******************************************************************************** 2025-01-03T11:41:01,259 Please avoid running ``setup.py`` directly. 2025-01-03T11:41:01,260 Instead, use pypa/build, pypa/installer or other 2025-01-03T11:41:01,261 standards-based tools. 2025-01-03T11:41:01,262 See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. 2025-01-03T11:41:01,263 ******************************************************************************** 2025-01-03T11:41:01,265 !! 2025-01-03T11:41:01,266 self.initialize_options() 2025-01-03T11:41:01,296 INFO:root:installing to build/bdist.linux-armv7l/wheel 2025-01-03T11:41:01,296 INFO:root:running install 2025-01-03T11:41:01,320 INFO:root:running install_lib 2025-01-03T11:41:01,348 INFO:root:creating build/bdist.linux-armv7l/wheel 2025-01-03T11:41:01,351 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer 2025-01-03T11:41:01,353 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/format 2025-01-03T11:41:01,355 INFO:root:copying build/lib/data_juicer/format/formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-03T11:41:01,358 INFO:root:copying build/lib/data_juicer/format/mixture_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-03T11:41:01,360 INFO:root:copying build/lib/data_juicer/format/text_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-03T11:41:01,362 INFO:root:copying build/lib/data_juicer/format/parquet_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-03T11:41:01,364 INFO:root:copying build/lib/data_juicer/format/empty_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-03T11:41:01,366 INFO:root:copying build/lib/data_juicer/format/load.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-03T11:41:01,368 INFO:root:copying build/lib/data_juicer/format/tsv_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-03T11:41:01,370 INFO:root:copying build/lib/data_juicer/format/csv_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-03T11:41:01,372 INFO:root:copying build/lib/data_juicer/format/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-03T11:41:01,374 INFO:root:copying build/lib/data_juicer/format/json_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-03T11:41:01,377 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/analysis 2025-01-03T11:41:01,378 INFO:root:copying build/lib/data_juicer/analysis/collector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-03T11:41:01,380 INFO:root:copying build/lib/data_juicer/analysis/measure.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-03T11:41:01,383 INFO:root:copying build/lib/data_juicer/analysis/draw.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-03T11:41:01,385 INFO:root:copying build/lib/data_juicer/analysis/column_wise_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-03T11:41:01,387 INFO:root:copying build/lib/data_juicer/analysis/diversity_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-03T11:41:01,390 INFO:root:copying build/lib/data_juicer/analysis/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-03T11:41:01,392 INFO:root:copying build/lib/data_juicer/analysis/overall_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-03T11:41:01,395 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/tools 2025-01-03T11:41:01,396 INFO:root:copying build/lib/data_juicer/tools/sandbox_starter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-01-03T11:41:01,398 INFO:root:copying build/lib/data_juicer/tools/dj_install.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-01-03T11:41:01,400 INFO:root:copying build/lib/data_juicer/tools/analyze_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-01-03T11:41:01,402 INFO:root:copying build/lib/data_juicer/tools/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-01-03T11:41:01,404 INFO:root:copying build/lib/data_juicer/tools/process_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-01-03T11:41:01,407 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/config 2025-01-03T11:41:01,408 INFO:root:copying build/lib/data_juicer/config/config.py -> build/bdist.linux-armv7l/wheel/./data_juicer/config 2025-01-03T11:41:01,411 INFO:root:copying build/lib/data_juicer/config/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/config 2025-01-03T11:41:01,414 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/utils 2025-01-03T11:41:01,415 INFO:root:copying build/lib/data_juicer/utils/compress.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,418 INFO:root:copying build/lib/data_juicer/utils/logger_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,420 INFO:root:copying build/lib/data_juicer/utils/model_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,424 INFO:root:copying build/lib/data_juicer/utils/resource_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,426 INFO:root:copying build/lib/data_juicer/utils/auto_install_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,428 INFO:root:copying build/lib/data_juicer/utils/process_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,430 INFO:root:copying build/lib/data_juicer/utils/availability_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,432 INFO:root:copying build/lib/data_juicer/utils/unittest_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,435 INFO:root:copying build/lib/data_juicer/utils/asset_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,436 INFO:root:copying build/lib/data_juicer/utils/cache_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,439 INFO:root:copying build/lib/data_juicer/utils/mm_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,441 INFO:root:copying build/lib/data_juicer/utils/constant.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,444 INFO:root:copying build/lib/data_juicer/utils/auto_install_mapping.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,447 INFO:root:copying build/lib/data_juicer/utils/file_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,449 INFO:root:copying build/lib/data_juicer/utils/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,451 INFO:root:copying build/lib/data_juicer/utils/lazy_loader.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,453 INFO:root:copying build/lib/data_juicer/utils/common_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,455 INFO:root:copying build/lib/data_juicer/utils/ckpt_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,457 INFO:root:copying build/lib/data_juicer/utils/registry.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,460 INFO:root:copying build/lib/data_juicer/utils/fingerprint_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-03T11:41:01,463 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops 2025-01-03T11:41:01,464 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/deduplicator 2025-01-03T11:41:01,466 INFO:root:copying build/lib/data_juicer/ops/deduplicator/video_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,468 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_basic_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,470 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_image_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,472 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_minhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,477 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,479 INFO:root:copying build/lib/data_juicer/ops/deduplicator/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,481 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_bts_minhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,483 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_simhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,485 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_video_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,487 INFO:root:copying build/lib/data_juicer/ops/deduplicator/image_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,489 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_document_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-03T11:41:01,491 INFO:root:copying build/lib/data_juicer/ops/base_op.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-01-03T11:41:01,495 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/grouper 2025-01-03T11:41:01,496 INFO:root:copying build/lib/data_juicer/ops/grouper/key_value_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-01-03T11:41:01,498 INFO:root:copying build/lib/data_juicer/ops/grouper/naive_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-01-03T11:41:01,500 INFO:root:copying build/lib/data_juicer/ops/grouper/naive_reverse_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-01-03T11:41:01,502 INFO:root:copying build/lib/data_juicer/ops/grouper/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-01-03T11:41:01,504 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/selector 2025-01-03T11:41:01,506 INFO:root:copying build/lib/data_juicer/ops/selector/topk_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-03T11:41:01,508 INFO:root:copying build/lib/data_juicer/ops/selector/frequency_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-03T11:41:01,510 INFO:root:copying build/lib/data_juicer/ops/selector/tags_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-03T11:41:01,512 INFO:root:copying build/lib/data_juicer/ops/selector/random_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-03T11:41:01,514 INFO:root:copying build/lib/data_juicer/ops/selector/range_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-03T11:41:01,517 INFO:root:copying build/lib/data_juicer/ops/selector/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-03T11:41:01,520 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/mapper 2025-01-03T11:41:01,522 INFO:root:copying build/lib/data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,524 INFO:root:copying build/lib/data_juicer/ops/mapper/generate_qa_from_examples_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,526 INFO:root:copying build/lib/data_juicer/ops/mapper/punctuation_normalization_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,529 INFO:root:copying build/lib/data_juicer/ops/mapper/image_diffusion_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,531 INFO:root:copying build/lib/data_juicer/ops/mapper/video_resize_resolution_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,533 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,536 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_non_chinese_character_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,537 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,540 INFO:root:copying build/lib/data_juicer/ops/mapper/image_face_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,542 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_response_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,544 INFO:root:copying build/lib/data_juicer/ops/mapper/query_intent_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,546 INFO:root:copying build/lib/data_juicer/ops/mapper/text_chunk_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,548 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_long_words_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,551 INFO:root:copying build/lib/data_juicer/ops/mapper/expand_macro_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,553 INFO:root:copying build/lib/data_juicer/ops/mapper/whitespace_normalization_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,555 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_sentiment_intensity_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,557 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_response_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,559 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_entity_attribute_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,562 INFO:root:copying build/lib/data_juicer/ops/mapper/fix_unicode_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,564 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_video_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,567 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_event_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,569 INFO:root:copying build/lib/data_juicer/ops/mapper/image_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,571 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_email_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,574 INFO:root:copying build/lib/data_juicer/ops/mapper/pair_preference_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,576 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_repeat_sentences_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,578 INFO:root:copying build/lib/data_juicer/ops/mapper/sentence_split_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,580 INFO:root:copying build/lib/data_juicer/ops/mapper/query_sentiment_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,582 INFO:root:copying build/lib/data_juicer/ops/mapper/python_lambda_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,584 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_entity_relation_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,587 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_intent_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,589 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,591 INFO:root:copying build/lib/data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,593 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_links_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,595 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_support_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,597 INFO:root:copying build/lib/data_juicer/ops/mapper/image_captioning_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,600 INFO:root:copying build/lib/data_juicer/ops/mapper/image_tagging_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,602 INFO:root:copying build/lib/data_juicer/ops/mapper/generate_qa_from_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,604 INFO:root:copying build/lib/data_juicer/ops/mapper/query_topic_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,606 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_scene_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,609 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_header_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,611 INFO:root:copying build/lib/data_juicer/ops/mapper/chinese_convert_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,613 INFO:root:copying build/lib/data_juicer/ops/mapper/video_extract_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,615 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_nickname_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,617 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_qa_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,620 INFO:root:copying build/lib/data_juicer/ops/mapper/video_tagging_from_audio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,622 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_copyright_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,624 INFO:root:copying build/lib/data_juicer/ops/mapper/nlpcda_zh_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,627 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_ip_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,628 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_key_frame_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,631 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_audio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,633 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_html_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,635 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_specific_chars_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,637 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_bibliography_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,639 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_sentiment_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,642 INFO:root:copying build/lib/data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,645 INFO:root:copying build/lib/data_juicer/ops/mapper/video_tagging_from_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,648 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_query_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,650 INFO:root:copying build/lib/data_juicer/ops/mapper/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,653 INFO:root:copying build/lib/data_juicer/ops/mapper/video_face_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,655 INFO:root:copying build/lib/data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,658 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_table_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,659 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_query_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,662 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_topic_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,664 INFO:root:copying build/lib/data_juicer/ops/mapper/video_remove_watermark_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,667 INFO:root:copying build/lib/data_juicer/ops/mapper/python_file_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,670 INFO:root:copying build/lib/data_juicer/ops/mapper/replace_content_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,672 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_keyword_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,674 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_qa_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,677 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_comments_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,678 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_duration_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,681 INFO:root:copying build/lib/data_juicer/ops/mapper/relation_identity_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,684 INFO:root:copying build/lib/data_juicer/ops/mapper/nlpaug_en_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-03T11:41:01,686 INFO:root:copying build/lib/data_juicer/ops/load.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-01-03T11:41:01,689 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/filter 2025-01-03T11:41:01,691 INFO:root:copying build/lib/data_juicer/ops/filter/video_watermark_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,693 INFO:root:copying build/lib/data_juicer/ops/filter/video_motion_score_raft_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,695 INFO:root:copying build/lib/data_juicer/ops/filter/image_nsfw_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,698 INFO:root:copying build/lib/data_juicer/ops/filter/words_num_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,699 INFO:root:copying build/lib/data_juicer/ops/filter/video_duration_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,702 INFO:root:copying build/lib/data_juicer/ops/filter/flagged_words_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,704 INFO:root:copying build/lib/data_juicer/ops/filter/perplexity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,706 INFO:root:copying build/lib/data_juicer/ops/filter/word_repetition_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,708 INFO:root:copying build/lib/data_juicer/ops/filter/phrase_grounding_recall_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,711 INFO:root:copying build/lib/data_juicer/ops/filter/specified_field_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,713 INFO:root:copying build/lib/data_juicer/ops/filter/video_aspect_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,715 INFO:root:copying build/lib/data_juicer/ops/filter/character_repetition_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,717 INFO:root:copying build/lib/data_juicer/ops/filter/image_text_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,720 INFO:root:copying build/lib/data_juicer/ops/filter/image_face_count_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,722 INFO:root:copying build/lib/data_juicer/ops/filter/image_face_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,725 INFO:root:copying build/lib/data_juicer/ops/filter/video_nsfw_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,727 INFO:root:copying build/lib/data_juicer/ops/filter/maximum_line_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,729 INFO:root:copying build/lib/data_juicer/ops/filter/stopwords_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,732 INFO:root:copying build/lib/data_juicer/ops/filter/video_motion_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,735 INFO:root:copying build/lib/data_juicer/ops/filter/text_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,738 INFO:root:copying build/lib/data_juicer/ops/filter/image_aspect_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,740 INFO:root:copying build/lib/data_juicer/ops/filter/audio_size_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,742 INFO:root:copying build/lib/data_juicer/ops/filter/suffix_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,744 INFO:root:copying build/lib/data_juicer/ops/filter/token_num_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,746 INFO:root:copying build/lib/data_juicer/ops/filter/video_aesthetics_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,748 INFO:root:copying build/lib/data_juicer/ops/filter/video_ocr_area_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,751 INFO:root:copying build/lib/data_juicer/ops/filter/audio_duration_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,752 INFO:root:copying build/lib/data_juicer/ops/filter/image_shape_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,755 INFO:root:copying build/lib/data_juicer/ops/filter/average_line_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,756 INFO:root:copying build/lib/data_juicer/ops/filter/image_size_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,758 INFO:root:copying build/lib/data_juicer/ops/filter/image_aesthetics_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,760 INFO:root:copying build/lib/data_juicer/ops/filter/image_watermark_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,762 INFO:root:copying build/lib/data_juicer/ops/filter/video_resolution_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,764 INFO:root:copying build/lib/data_juicer/ops/filter/language_id_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,766 INFO:root:copying build/lib/data_juicer/ops/filter/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,768 INFO:root:copying build/lib/data_juicer/ops/filter/text_entity_dependency_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,771 INFO:root:copying build/lib/data_juicer/ops/filter/image_pair_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,774 INFO:root:copying build/lib/data_juicer/ops/filter/text_action_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,776 INFO:root:copying build/lib/data_juicer/ops/filter/image_text_matching_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,778 INFO:root:copying build/lib/data_juicer/ops/filter/video_frames_text_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,781 INFO:root:copying build/lib/data_juicer/ops/filter/special_characters_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,783 INFO:root:copying build/lib/data_juicer/ops/filter/video_tagging_from_frames_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,785 INFO:root:copying build/lib/data_juicer/ops/filter/specified_numeric_field_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,787 INFO:root:copying build/lib/data_juicer/ops/filter/alphanumeric_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,798 INFO:root:copying build/lib/data_juicer/ops/filter/audio_nmf_snr_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-03T11:41:01,800 INFO:root:copying build/lib/data_juicer/ops/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-01-03T11:41:01,802 INFO:root:copying build/lib/data_juicer/ops/op_fusion.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-01-03T11:41:01,806 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/common 2025-01-03T11:41:01,807 INFO:root:copying build/lib/data_juicer/ops/common/special_characters.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2025-01-03T11:41:01,810 INFO:root:copying build/lib/data_juicer/ops/common/helper_func.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2025-01-03T11:41:01,812 INFO:root:copying build/lib/data_juicer/ops/common/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2025-01-03T11:41:01,815 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/aggregator 2025-01-03T11:41:01,816 INFO:root:copying build/lib/data_juicer/ops/aggregator/entity_attribute_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-01-03T11:41:01,819 INFO:root:copying build/lib/data_juicer/ops/aggregator/meta_tags_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-01-03T11:41:01,821 INFO:root:copying build/lib/data_juicer/ops/aggregator/nested_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-01-03T11:41:01,824 INFO:root:copying build/lib/data_juicer/ops/aggregator/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-01-03T11:41:01,826 INFO:root:copying build/lib/data_juicer/ops/aggregator/most_relavant_entities_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-01-03T11:41:01,829 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/core 2025-01-03T11:41:01,830 INFO:root:copying build/lib/data_juicer/core/tracer.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-03T11:41:01,833 INFO:root:copying build/lib/data_juicer/core/exporter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-03T11:41:01,835 INFO:root:copying build/lib/data_juicer/core/analyzer.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-03T11:41:01,838 INFO:root:copying build/lib/data_juicer/core/ray_executor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-03T11:41:01,840 INFO:root:copying build/lib/data_juicer/core/data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-03T11:41:01,844 INFO:root:copying build/lib/data_juicer/core/executor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-03T11:41:01,846 INFO:root:copying build/lib/data_juicer/core/monitor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-03T11:41:01,848 INFO:root:copying build/lib/data_juicer/core/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-03T11:41:01,851 INFO:root:copying build/lib/data_juicer/core/ray_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-03T11:41:01,853 INFO:root:copying build/lib/data_juicer/core/adapter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-03T11:41:01,855 INFO:root:copying build/lib/data_juicer/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer 2025-01-03T11:41:01,857 INFO:root:running install_egg_info 2025-01-03T11:41:01,892 INFO:root:running egg_info 2025-01-03T11:41:01,920 INFO:root:writing py_data_juicer.egg-info/PKG-INFO 2025-01-03T11:41:01,925 INFO:root:writing dependency_links to py_data_juicer.egg-info/dependency_links.txt 2025-01-03T11:41:01,926 INFO:root:writing entry points to py_data_juicer.egg-info/entry_points.txt 2025-01-03T11:41:01,928 INFO:root:writing requirements to py_data_juicer.egg-info/requires.txt 2025-01-03T11:41:01,929 INFO:root:writing top-level names to py_data_juicer.egg-info/top_level.txt 2025-01-03T11:41:01,981 INFO:root:reading manifest file 'py_data_juicer.egg-info/SOURCES.txt' 2025-01-03T11:41:01,993 INFO:root:adding license file 'LICENSE' 2025-01-03T11:41:02,005 INFO:root:writing manifest file 'py_data_juicer.egg-info/SOURCES.txt' 2025-01-03T11:41:02,007 INFO:root:Copying py_data_juicer.egg-info to build/bdist.linux-armv7l/wheel/./py_data_juicer-1.0.3-py3.11.egg-info 2025-01-03T11:41:02,020 INFO:root:running install_scripts 2025-01-03T11:41:02,042 INFO:root:creating build/bdist.linux-armv7l/wheel/py_data_juicer-1.0.3.dist-info/WHEEL 2025-01-03T11:41:02,044 INFO:wheel:creating '/tmp/pip-wheel-1ew9idva/py_data_juicer-1.0.3-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2025-01-03T11:41:02,047 INFO:wheel:adding 'data_juicer/__init__.py' 2025-01-03T11:41:02,048 INFO:wheel:adding 'data_juicer/analysis/__init__.py' 2025-01-03T11:41:02,050 INFO:wheel:adding 'data_juicer/analysis/collector.py' 2025-01-03T11:41:02,052 INFO:wheel:adding 'data_juicer/analysis/column_wise_analysis.py' 2025-01-03T11:41:02,054 INFO:wheel:adding 'data_juicer/analysis/diversity_analysis.py' 2025-01-03T11:41:02,055 INFO:wheel:adding 'data_juicer/analysis/draw.py' 2025-01-03T11:41:02,056 INFO:wheel:adding 'data_juicer/analysis/measure.py' 2025-01-03T11:41:02,058 INFO:wheel:adding 'data_juicer/analysis/overall_analysis.py' 2025-01-03T11:41:02,060 INFO:wheel:adding 'data_juicer/config/__init__.py' 2025-01-03T11:41:02,064 INFO:wheel:adding 'data_juicer/config/config.py' 2025-01-03T11:41:02,066 INFO:wheel:adding 'data_juicer/core/__init__.py' 2025-01-03T11:41:02,068 INFO:wheel:adding 'data_juicer/core/adapter.py' 2025-01-03T11:41:02,070 INFO:wheel:adding 'data_juicer/core/analyzer.py' 2025-01-03T11:41:02,072 INFO:wheel:adding 'data_juicer/core/data.py' 2025-01-03T11:41:02,074 INFO:wheel:adding 'data_juicer/core/executor.py' 2025-01-03T11:41:02,076 INFO:wheel:adding 'data_juicer/core/exporter.py' 2025-01-03T11:41:02,079 INFO:wheel:adding 'data_juicer/core/monitor.py' 2025-01-03T11:41:02,080 INFO:wheel:adding 'data_juicer/core/ray_data.py' 2025-01-03T11:41:02,082 INFO:wheel:adding 'data_juicer/core/ray_executor.py' 2025-01-03T11:41:02,084 INFO:wheel:adding 'data_juicer/core/tracer.py' 2025-01-03T11:41:02,086 INFO:wheel:adding 'data_juicer/format/__init__.py' 2025-01-03T11:41:02,087 INFO:wheel:adding 'data_juicer/format/csv_formatter.py' 2025-01-03T11:41:02,089 INFO:wheel:adding 'data_juicer/format/empty_formatter.py' 2025-01-03T11:41:02,090 INFO:wheel:adding 'data_juicer/format/formatter.py' 2025-01-03T11:41:02,092 INFO:wheel:adding 'data_juicer/format/json_formatter.py' 2025-01-03T11:41:02,093 INFO:wheel:adding 'data_juicer/format/load.py' 2025-01-03T11:41:02,094 INFO:wheel:adding 'data_juicer/format/mixture_formatter.py' 2025-01-03T11:41:02,096 INFO:wheel:adding 'data_juicer/format/parquet_formatter.py' 2025-01-03T11:41:02,097 INFO:wheel:adding 'data_juicer/format/text_formatter.py' 2025-01-03T11:41:02,099 INFO:wheel:adding 'data_juicer/format/tsv_formatter.py' 2025-01-03T11:41:02,100 INFO:wheel:adding 'data_juicer/ops/__init__.py' 2025-01-03T11:41:02,103 INFO:wheel:adding 'data_juicer/ops/base_op.py' 2025-01-03T11:41:02,104 INFO:wheel:adding 'data_juicer/ops/load.py' 2025-01-03T11:41:02,106 INFO:wheel:adding 'data_juicer/ops/op_fusion.py' 2025-01-03T11:41:02,108 INFO:wheel:adding 'data_juicer/ops/aggregator/__init__.py' 2025-01-03T11:41:02,109 INFO:wheel:adding 'data_juicer/ops/aggregator/entity_attribute_aggregator.py' 2025-01-03T11:41:02,111 INFO:wheel:adding 'data_juicer/ops/aggregator/meta_tags_aggregator.py' 2025-01-03T11:41:02,113 INFO:wheel:adding 'data_juicer/ops/aggregator/most_relavant_entities_aggregator.py' 2025-01-03T11:41:02,115 INFO:wheel:adding 'data_juicer/ops/aggregator/nested_aggregator.py' 2025-01-03T11:41:02,117 INFO:wheel:adding 'data_juicer/ops/common/__init__.py' 2025-01-03T11:41:02,118 INFO:wheel:adding 'data_juicer/ops/common/helper_func.py' 2025-01-03T11:41:02,120 INFO:wheel:adding 'data_juicer/ops/common/special_characters.py' 2025-01-03T11:41:02,121 INFO:wheel:adding 'data_juicer/ops/deduplicator/__init__.py' 2025-01-03T11:41:02,123 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_deduplicator.py' 2025-01-03T11:41:02,125 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_minhash_deduplicator.py' 2025-01-03T11:41:02,127 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_simhash_deduplicator.py' 2025-01-03T11:41:02,128 INFO:wheel:adding 'data_juicer/ops/deduplicator/image_deduplicator.py' 2025-01-03T11:41:02,130 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_basic_deduplicator.py' 2025-01-03T11:41:02,132 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_bts_minhash_deduplicator.py' 2025-01-03T11:41:02,134 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_document_deduplicator.py' 2025-01-03T11:41:02,135 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_image_deduplicator.py' 2025-01-03T11:41:02,136 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_video_deduplicator.py' 2025-01-03T11:41:02,138 INFO:wheel:adding 'data_juicer/ops/deduplicator/video_deduplicator.py' 2025-01-03T11:41:02,140 INFO:wheel:adding 'data_juicer/ops/filter/__init__.py' 2025-01-03T11:41:02,142 INFO:wheel:adding 'data_juicer/ops/filter/alphanumeric_filter.py' 2025-01-03T11:41:02,144 INFO:wheel:adding 'data_juicer/ops/filter/audio_duration_filter.py' 2025-01-03T11:41:02,145 INFO:wheel:adding 'data_juicer/ops/filter/audio_nmf_snr_filter.py' 2025-01-03T11:41:02,146 INFO:wheel:adding 'data_juicer/ops/filter/audio_size_filter.py' 2025-01-03T11:41:02,147 INFO:wheel:adding 'data_juicer/ops/filter/average_line_length_filter.py' 2025-01-03T11:41:02,149 INFO:wheel:adding 'data_juicer/ops/filter/character_repetition_filter.py' 2025-01-03T11:41:02,150 INFO:wheel:adding 'data_juicer/ops/filter/flagged_words_filter.py' 2025-01-03T11:41:02,152 INFO:wheel:adding 'data_juicer/ops/filter/image_aesthetics_filter.py' 2025-01-03T11:41:02,153 INFO:wheel:adding 'data_juicer/ops/filter/image_aspect_ratio_filter.py' 2025-01-03T11:41:02,155 INFO:wheel:adding 'data_juicer/ops/filter/image_face_count_filter.py' 2025-01-03T11:41:02,156 INFO:wheel:adding 'data_juicer/ops/filter/image_face_ratio_filter.py' 2025-01-03T11:41:02,158 INFO:wheel:adding 'data_juicer/ops/filter/image_nsfw_filter.py' 2025-01-03T11:41:02,160 INFO:wheel:adding 'data_juicer/ops/filter/image_pair_similarity_filter.py' 2025-01-03T11:41:02,161 INFO:wheel:adding 'data_juicer/ops/filter/image_shape_filter.py' 2025-01-03T11:41:02,163 INFO:wheel:adding 'data_juicer/ops/filter/image_size_filter.py' 2025-01-03T11:41:02,164 INFO:wheel:adding 'data_juicer/ops/filter/image_text_matching_filter.py' 2025-01-03T11:41:02,166 INFO:wheel:adding 'data_juicer/ops/filter/image_text_similarity_filter.py' 2025-01-03T11:41:02,167 INFO:wheel:adding 'data_juicer/ops/filter/image_watermark_filter.py' 2025-01-03T11:41:02,169 INFO:wheel:adding 'data_juicer/ops/filter/language_id_score_filter.py' 2025-01-03T11:41:02,171 INFO:wheel:adding 'data_juicer/ops/filter/maximum_line_length_filter.py' 2025-01-03T11:41:02,172 INFO:wheel:adding 'data_juicer/ops/filter/perplexity_filter.py' 2025-01-03T11:41:02,174 INFO:wheel:adding 'data_juicer/ops/filter/phrase_grounding_recall_filter.py' 2025-01-03T11:41:02,177 INFO:wheel:adding 'data_juicer/ops/filter/special_characters_filter.py' 2025-01-03T11:41:02,179 INFO:wheel:adding 'data_juicer/ops/filter/specified_field_filter.py' 2025-01-03T11:41:02,180 INFO:wheel:adding 'data_juicer/ops/filter/specified_numeric_field_filter.py' 2025-01-03T11:41:02,182 INFO:wheel:adding 'data_juicer/ops/filter/stopwords_filter.py' 2025-01-03T11:41:02,183 INFO:wheel:adding 'data_juicer/ops/filter/suffix_filter.py' 2025-01-03T11:41:02,184 INFO:wheel:adding 'data_juicer/ops/filter/text_action_filter.py' 2025-01-03T11:41:02,186 INFO:wheel:adding 'data_juicer/ops/filter/text_entity_dependency_filter.py' 2025-01-03T11:41:02,187 INFO:wheel:adding 'data_juicer/ops/filter/text_length_filter.py' 2025-01-03T11:41:02,188 INFO:wheel:adding 'data_juicer/ops/filter/token_num_filter.py' 2025-01-03T11:41:02,190 INFO:wheel:adding 'data_juicer/ops/filter/video_aesthetics_filter.py' 2025-01-03T11:41:02,192 INFO:wheel:adding 'data_juicer/ops/filter/video_aspect_ratio_filter.py' 2025-01-03T11:41:02,194 INFO:wheel:adding 'data_juicer/ops/filter/video_duration_filter.py' 2025-01-03T11:41:02,196 INFO:wheel:adding 'data_juicer/ops/filter/video_frames_text_similarity_filter.py' 2025-01-03T11:41:02,197 INFO:wheel:adding 'data_juicer/ops/filter/video_motion_score_filter.py' 2025-01-03T11:41:02,199 INFO:wheel:adding 'data_juicer/ops/filter/video_motion_score_raft_filter.py' 2025-01-03T11:41:02,200 INFO:wheel:adding 'data_juicer/ops/filter/video_nsfw_filter.py' 2025-01-03T11:41:02,202 INFO:wheel:adding 'data_juicer/ops/filter/video_ocr_area_ratio_filter.py' 2025-01-03T11:41:02,204 INFO:wheel:adding 'data_juicer/ops/filter/video_resolution_filter.py' 2025-01-03T11:41:02,205 INFO:wheel:adding 'data_juicer/ops/filter/video_tagging_from_frames_filter.py' 2025-01-03T11:41:02,207 INFO:wheel:adding 'data_juicer/ops/filter/video_watermark_filter.py' 2025-01-03T11:41:02,208 INFO:wheel:adding 'data_juicer/ops/filter/word_repetition_filter.py' 2025-01-03T11:41:02,210 INFO:wheel:adding 'data_juicer/ops/filter/words_num_filter.py' 2025-01-03T11:41:02,211 INFO:wheel:adding 'data_juicer/ops/grouper/__init__.py' 2025-01-03T11:41:02,212 INFO:wheel:adding 'data_juicer/ops/grouper/key_value_grouper.py' 2025-01-03T11:41:02,214 INFO:wheel:adding 'data_juicer/ops/grouper/naive_grouper.py' 2025-01-03T11:41:02,215 INFO:wheel:adding 'data_juicer/ops/grouper/naive_reverse_grouper.py' 2025-01-03T11:41:02,218 INFO:wheel:adding 'data_juicer/ops/mapper/__init__.py' 2025-01-03T11:41:02,220 INFO:wheel:adding 'data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py' 2025-01-03T11:41:02,222 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_qa_mapper.py' 2025-01-03T11:41:02,223 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_query_mapper.py' 2025-01-03T11:41:02,224 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_response_mapper.py' 2025-01-03T11:41:02,225 INFO:wheel:adding 'data_juicer/ops/mapper/chinese_convert_mapper.py' 2025-01-03T11:41:02,227 INFO:wheel:adding 'data_juicer/ops/mapper/clean_copyright_mapper.py' 2025-01-03T11:41:02,228 INFO:wheel:adding 'data_juicer/ops/mapper/clean_email_mapper.py' 2025-01-03T11:41:02,229 INFO:wheel:adding 'data_juicer/ops/mapper/clean_html_mapper.py' 2025-01-03T11:41:02,231 INFO:wheel:adding 'data_juicer/ops/mapper/clean_ip_mapper.py' 2025-01-03T11:41:02,232 INFO:wheel:adding 'data_juicer/ops/mapper/clean_links_mapper.py' 2025-01-03T11:41:02,234 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_intent_detection_mapper.py' 2025-01-03T11:41:02,236 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_sentiment_detection_mapper.py' 2025-01-03T11:41:02,238 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_sentiment_intensity_mapper.py' 2025-01-03T11:41:02,240 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_topic_detection_mapper.py' 2025-01-03T11:41:02,242 INFO:wheel:adding 'data_juicer/ops/mapper/expand_macro_mapper.py' 2025-01-03T11:41:02,243 INFO:wheel:adding 'data_juicer/ops/mapper/extract_entity_attribute_mapper.py' 2025-01-03T11:41:02,246 INFO:wheel:adding 'data_juicer/ops/mapper/extract_entity_relation_mapper.py' 2025-01-03T11:41:02,248 INFO:wheel:adding 'data_juicer/ops/mapper/extract_event_mapper.py' 2025-01-03T11:41:02,250 INFO:wheel:adding 'data_juicer/ops/mapper/extract_keyword_mapper.py' 2025-01-03T11:41:02,252 INFO:wheel:adding 'data_juicer/ops/mapper/extract_nickname_mapper.py' 2025-01-03T11:41:02,254 INFO:wheel:adding 'data_juicer/ops/mapper/extract_support_text_mapper.py' 2025-01-03T11:41:02,255 INFO:wheel:adding 'data_juicer/ops/mapper/fix_unicode_mapper.py' 2025-01-03T11:41:02,257 INFO:wheel:adding 'data_juicer/ops/mapper/generate_qa_from_examples_mapper.py' 2025-01-03T11:41:02,259 INFO:wheel:adding 'data_juicer/ops/mapper/generate_qa_from_text_mapper.py' 2025-01-03T11:41:02,261 INFO:wheel:adding 'data_juicer/ops/mapper/image_blur_mapper.py' 2025-01-03T11:41:02,263 INFO:wheel:adding 'data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py' 2025-01-03T11:41:02,265 INFO:wheel:adding 'data_juicer/ops/mapper/image_captioning_mapper.py' 2025-01-03T11:41:02,267 INFO:wheel:adding 'data_juicer/ops/mapper/image_diffusion_mapper.py' 2025-01-03T11:41:02,269 INFO:wheel:adding 'data_juicer/ops/mapper/image_face_blur_mapper.py' 2025-01-03T11:41:02,270 INFO:wheel:adding 'data_juicer/ops/mapper/image_tagging_mapper.py' 2025-01-03T11:41:02,272 INFO:wheel:adding 'data_juicer/ops/mapper/nlpaug_en_mapper.py' 2025-01-03T11:41:02,273 INFO:wheel:adding 'data_juicer/ops/mapper/nlpcda_zh_mapper.py' 2025-01-03T11:41:02,275 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_qa_mapper.py' 2025-01-03T11:41:02,276 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_query_mapper.py' 2025-01-03T11:41:02,277 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_response_mapper.py' 2025-01-03T11:41:02,278 INFO:wheel:adding 'data_juicer/ops/mapper/pair_preference_mapper.py' 2025-01-03T11:41:02,279 INFO:wheel:adding 'data_juicer/ops/mapper/punctuation_normalization_mapper.py' 2025-01-03T11:41:02,281 INFO:wheel:adding 'data_juicer/ops/mapper/python_file_mapper.py' 2025-01-03T11:41:02,282 INFO:wheel:adding 'data_juicer/ops/mapper/python_lambda_mapper.py' 2025-01-03T11:41:02,283 INFO:wheel:adding 'data_juicer/ops/mapper/query_intent_detection_mapper.py' 2025-01-03T11:41:02,284 INFO:wheel:adding 'data_juicer/ops/mapper/query_sentiment_detection_mapper.py' 2025-01-03T11:41:02,286 INFO:wheel:adding 'data_juicer/ops/mapper/query_topic_detection_mapper.py' 2025-01-03T11:41:02,287 INFO:wheel:adding 'data_juicer/ops/mapper/relation_identity_mapper.py' 2025-01-03T11:41:02,289 INFO:wheel:adding 'data_juicer/ops/mapper/remove_bibliography_mapper.py' 2025-01-03T11:41:02,290 INFO:wheel:adding 'data_juicer/ops/mapper/remove_comments_mapper.py' 2025-01-03T11:41:02,291 INFO:wheel:adding 'data_juicer/ops/mapper/remove_header_mapper.py' 2025-01-03T11:41:02,292 INFO:wheel:adding 'data_juicer/ops/mapper/remove_long_words_mapper.py' 2025-01-03T11:41:02,294 INFO:wheel:adding 'data_juicer/ops/mapper/remove_non_chinese_character_mapper.py' 2025-01-03T11:41:02,295 INFO:wheel:adding 'data_juicer/ops/mapper/remove_repeat_sentences_mapper.py' 2025-01-03T11:41:02,296 INFO:wheel:adding 'data_juicer/ops/mapper/remove_specific_chars_mapper.py' 2025-01-03T11:41:02,298 INFO:wheel:adding 'data_juicer/ops/mapper/remove_table_text_mapper.py' 2025-01-03T11:41:02,299 INFO:wheel:adding 'data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py' 2025-01-03T11:41:02,300 INFO:wheel:adding 'data_juicer/ops/mapper/replace_content_mapper.py' 2025-01-03T11:41:02,302 INFO:wheel:adding 'data_juicer/ops/mapper/sentence_split_mapper.py' 2025-01-03T11:41:02,303 INFO:wheel:adding 'data_juicer/ops/mapper/text_chunk_mapper.py' 2025-01-03T11:41:02,305 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_audio_mapper.py' 2025-01-03T11:41:02,307 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_frames_mapper.py' 2025-01-03T11:41:02,309 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py' 2025-01-03T11:41:02,311 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_video_mapper.py' 2025-01-03T11:41:02,313 INFO:wheel:adding 'data_juicer/ops/mapper/video_extract_frames_mapper.py' 2025-01-03T11:41:02,314 INFO:wheel:adding 'data_juicer/ops/mapper/video_face_blur_mapper.py' 2025-01-03T11:41:02,316 INFO:wheel:adding 'data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py' 2025-01-03T11:41:02,318 INFO:wheel:adding 'data_juicer/ops/mapper/video_remove_watermark_mapper.py' 2025-01-03T11:41:02,319 INFO:wheel:adding 'data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py' 2025-01-03T11:41:02,321 INFO:wheel:adding 'data_juicer/ops/mapper/video_resize_resolution_mapper.py' 2025-01-03T11:41:02,323 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_duration_mapper.py' 2025-01-03T11:41:02,324 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_key_frame_mapper.py' 2025-01-03T11:41:02,326 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_scene_mapper.py' 2025-01-03T11:41:02,327 INFO:wheel:adding 'data_juicer/ops/mapper/video_tagging_from_audio_mapper.py' 2025-01-03T11:41:02,329 INFO:wheel:adding 'data_juicer/ops/mapper/video_tagging_from_frames_mapper.py' 2025-01-03T11:41:02,330 INFO:wheel:adding 'data_juicer/ops/mapper/whitespace_normalization_mapper.py' 2025-01-03T11:41:02,332 INFO:wheel:adding 'data_juicer/ops/selector/__init__.py' 2025-01-03T11:41:02,333 INFO:wheel:adding 'data_juicer/ops/selector/frequency_specified_field_selector.py' 2025-01-03T11:41:02,334 INFO:wheel:adding 'data_juicer/ops/selector/random_selector.py' 2025-01-03T11:41:02,336 INFO:wheel:adding 'data_juicer/ops/selector/range_specified_field_selector.py' 2025-01-03T11:41:02,337 INFO:wheel:adding 'data_juicer/ops/selector/tags_specified_field_selector.py' 2025-01-03T11:41:02,338 INFO:wheel:adding 'data_juicer/ops/selector/topk_specified_field_selector.py' 2025-01-03T11:41:02,340 INFO:wheel:adding 'data_juicer/tools/__init__.py' 2025-01-03T11:41:02,341 INFO:wheel:adding 'data_juicer/tools/analyze_data.py' 2025-01-03T11:41:02,342 INFO:wheel:adding 'data_juicer/tools/dj_install.py' 2025-01-03T11:41:02,343 INFO:wheel:adding 'data_juicer/tools/process_data.py' 2025-01-03T11:41:02,344 INFO:wheel:adding 'data_juicer/tools/sandbox_starter.py' 2025-01-03T11:41:02,346 INFO:wheel:adding 'data_juicer/utils/__init__.py' 2025-01-03T11:41:02,347 INFO:wheel:adding 'data_juicer/utils/asset_utils.py' 2025-01-03T11:41:02,349 INFO:wheel:adding 'data_juicer/utils/auto_install_mapping.py' 2025-01-03T11:41:02,350 INFO:wheel:adding 'data_juicer/utils/auto_install_utils.py' 2025-01-03T11:41:02,351 INFO:wheel:adding 'data_juicer/utils/availability_utils.py' 2025-01-03T11:41:02,352 INFO:wheel:adding 'data_juicer/utils/cache_utils.py' 2025-01-03T11:41:02,354 INFO:wheel:adding 'data_juicer/utils/ckpt_utils.py' 2025-01-03T11:41:02,355 INFO:wheel:adding 'data_juicer/utils/common_utils.py' 2025-01-03T11:41:02,358 INFO:wheel:adding 'data_juicer/utils/compress.py' 2025-01-03T11:41:02,360 INFO:wheel:adding 'data_juicer/utils/constant.py' 2025-01-03T11:41:02,361 INFO:wheel:adding 'data_juicer/utils/file_utils.py' 2025-01-03T11:41:02,363 INFO:wheel:adding 'data_juicer/utils/fingerprint_utils.py' 2025-01-03T11:41:02,365 INFO:wheel:adding 'data_juicer/utils/lazy_loader.py' 2025-01-03T11:41:02,366 INFO:wheel:adding 'data_juicer/utils/logger_utils.py' 2025-01-03T11:41:02,370 INFO:wheel:adding 'data_juicer/utils/mm_utils.py' 2025-01-03T11:41:02,374 INFO:wheel:adding 'data_juicer/utils/model_utils.py' 2025-01-03T11:41:02,376 INFO:wheel:adding 'data_juicer/utils/process_utils.py' 2025-01-03T11:41:02,378 INFO:wheel:adding 'data_juicer/utils/registry.py' 2025-01-03T11:41:02,379 INFO:wheel:adding 'data_juicer/utils/resource_utils.py' 2025-01-03T11:41:02,380 INFO:wheel:adding 'data_juicer/utils/unittest_utils.py' 2025-01-03T11:41:02,384 INFO:wheel:adding 'py_data_juicer-1.0.3.dist-info/LICENSE' 2025-01-03T11:41:02,389 INFO:wheel:adding 'py_data_juicer-1.0.3.dist-info/METADATA' 2025-01-03T11:41:02,390 INFO:wheel:adding 'py_data_juicer-1.0.3.dist-info/WHEEL' 2025-01-03T11:41:02,391 INFO:wheel:adding 'py_data_juicer-1.0.3.dist-info/entry_points.txt' 2025-01-03T11:41:02,392 INFO:wheel:adding 'py_data_juicer-1.0.3.dist-info/top_level.txt' 2025-01-03T11:41:02,395 INFO:wheel:adding 'py_data_juicer-1.0.3.dist-info/RECORD' 2025-01-03T11:41:02,405 INFO:root:removing build/bdist.linux-armv7l/wheel 2025-01-03T11:41:02,546 Building wheel for py-data-juicer (setup.py): finished with status 'done' 2025-01-03T11:41:02,552 Created wheel for py-data-juicer: filename=py_data_juicer-1.0.3-py3-none-any.whl size=389135 sha256=ec09d1fb68352b5fb204a2cab5cac661f02323ef0cbc6a52833ea83418e55a3a 2025-01-03T11:41:02,554 Stored in directory: /tmp/pip-ephem-wheel-cache-_r962h76/wheels/1e/b8/3f/9068f9002e15512d3ebb88618d1cbb5a3615ba23033b46c533 2025-01-03T11:41:02,579 Successfully built py-data-juicer 2025-01-03T11:41:02,597 Removed build tracker: '/tmp/pip-build-tracker-gufl43z0'