2025-01-17T10:46:37,539 Created temporary directory: /tmp/pip-build-tracker-9kl3el4x 2025-01-17T10:46:37,541 Initialized build tracking at /tmp/pip-build-tracker-9kl3el4x 2025-01-17T10:46:37,541 Created build tracker: /tmp/pip-build-tracker-9kl3el4x 2025-01-17T10:46:37,542 Entered build tracker: /tmp/pip-build-tracker-9kl3el4x 2025-01-17T10:46:37,543 Created temporary directory: /tmp/pip-wheel-fz4yxowk 2025-01-17T10:46:37,546 Created temporary directory: /tmp/pip-ephem-wheel-cache-89sxn7kp 2025-01-17T10:46:37,590 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2025-01-17T10:46:37,593 2 location(s) to search for versions of py-data-juicer: 2025-01-17T10:46:37,593 * https://pypi.org/simple/py-data-juicer/ 2025-01-17T10:46:37,593 * https://www.piwheels.org/simple/py-data-juicer/ 2025-01-17T10:46:37,593 Fetching project page and analyzing links: https://pypi.org/simple/py-data-juicer/ 2025-01-17T10:46:37,594 Getting page https://pypi.org/simple/py-data-juicer/ 2025-01-17T10:46:37,596 Found index url https://pypi.org/simple/ 2025-01-17T10:46:37,814 Fetched page https://pypi.org/simple/py-data-juicer/ as application/vnd.pypi.simple.v1+json 2025-01-17T10:46:37,819 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/e3/d3/d0724d922e7c55a0664485fdf642124bdcd801df2697e29c9463c4360958/py_data_juicer-0.1.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-17T10:46:37,819 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/fd/c6/e1428310bf534319ddc2e362c6eee21985f097e70cedc0e9cd3e914de826/py_data_juicer-0.1.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-17T10:46:37,820 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/67/70/731e349d2a92bf59a767230b956665dd27ea24a7a948d4a1710154d77a24/py_data_juicer-0.1.2-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-17T10:46:37,820 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/66/08/f584efbdf8277a061ce73c9befc33fec813cbd08020761af2380dd331692/py_data_juicer-0.1.3-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-17T10:46:37,821 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/69/20/9862dfe7a94f10caa0e6387834d1265420613345c7adadd41a3ced7230e8/py_data_juicer-0.2.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-17T10:46:37,822 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/05/34/33fb401d350f1a66cdaefc224419f9d01484722829f5b0c71ef59a8365a4/py_data_juicer-1.0.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-17T10:46:37,822 Found link https://files.pythonhosted.org/packages/49/e9/cfb994255490c36554048e0b3956858e61f5a75ffacfde20849f31f04ef8/py_data_juicer-1.0.0.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.0 2025-01-17T10:46:37,823 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/27/37/150e2198f14349fdd6b647a63feaede809e21dbfc4617aba487f18049828/py_data_juicer-1.0.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-17T10:46:37,824 Found link https://files.pythonhosted.org/packages/4a/60/dadcbe4337a76d8f98022b60d142ac31073d568c9e06a5461b3014d16fc6/py_data_juicer-1.0.1.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.1 2025-01-17T10:46:37,824 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/10/dd/f9cadd6ed2f19c4f94e61a3fd4bb5bb2029c20b2cf61ec86c3a05f89f74b/py_data_juicer-1.0.2-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-17T10:46:37,825 Found link https://files.pythonhosted.org/packages/95/8f/c0447ce091ca0b86283e8074a3cb350572a5436ca13dceb288887a0de332/py_data_juicer-1.0.2.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.2 2025-01-17T10:46:37,826 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/1f/dd/0f1804c5cfce50c52c8b60a1b710f30d893f681e42a6acc41bfd29a20059/py_data_juicer-1.0.3-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-17T10:46:37,826 Found link https://files.pythonhosted.org/packages/6e/d0/eefb1ca00cd4c8e62c0e8f62a1b5357b961ce50c5d2647107f51f68bf128/py_data_juicer-1.0.3.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.3 2025-01-17T10:46:37,827 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/5b/06/5df5581f724d49731b91db0b7c65fdde739358617af9e7ba9916c43f2a76/py_data_juicer-1.1.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-01-17T10:46:37,828 Found link https://files.pythonhosted.org/packages/69/43/f20a50fdbfc0ba44a4816d3c7f9bfcfaf55682ffebf78a10eeff6b40feeb/py_data_juicer-1.1.0.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.1.0 2025-01-17T10:46:37,828 Fetching project page and analyzing links: https://www.piwheels.org/simple/py-data-juicer/ 2025-01-17T10:46:37,829 Getting page https://www.piwheels.org/simple/py-data-juicer/ 2025-01-17T10:46:37,830 Found index url https://www.piwheels.org/simple/ 2025-01-17T10:46:37,992 Fetched page https://www.piwheels.org/simple/py-data-juicer/ as text/html 2025-01-17T10:46:37,994 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.3-py3-none-any.whl#sha256=ec09d1fb68352b5fb204a2cab5cac661f02323ef0cbc6a52833ea83418e55a3a (from https://www.piwheels.org/simple/py-data-juicer/) 2025-01-17T10:46:37,995 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.2-py3-none-any.whl#sha256=69bc91a28885bd8df218510cd9c0198397f76eaab0c7fbf9726181da70734781 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-01-17T10:46:37,995 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.1-py3-none-any.whl#sha256=2396c39923ab13a9fe17fe6523734f2ff41b3edb2c97fcc70539a5761e0465f8 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-01-17T10:46:37,996 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.0-py3-none-any.whl#sha256=7cd007456933447474ec9faf5357b741a63e7a0c0dc6230e40bf4498eb1ef554 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-01-17T10:46:37,997 Skipping link: not a file: https://www.piwheels.org/simple/py-data-juicer/ 2025-01-17T10:46:37,997 Skipping link: not a file: https://pypi.org/simple/py-data-juicer/ 2025-01-17T10:46:38,017 Given no hashes to check 1 links for project 'py-data-juicer': discarding no candidates 2025-01-17T10:46:38,019 Collecting py-data-juicer==1.1.0 2025-01-17T10:46:38,021 Created temporary directory: /tmp/pip-unpack-7pfxr8s2 2025-01-17T10:46:38,266 Downloading py_data_juicer-1.1.0.tar.gz (282 kB) 2025-01-17T10:46:38,780 Added py-data-juicer==1.1.0 from https://files.pythonhosted.org/packages/69/43/f20a50fdbfc0ba44a4816d3c7f9bfcfaf55682ffebf78a10eeff6b40feeb/py_data_juicer-1.1.0.tar.gz to build tracker '/tmp/pip-build-tracker-9kl3el4x' 2025-01-17T10:46:38,782 Running setup.py (path:/tmp/pip-wheel-fz4yxowk/py-data-juicer_5922be90fbb54d05a6b7fced05bd0658/setup.py) egg_info for package py-data-juicer 2025-01-17T10:46:38,783 Created temporary directory: /tmp/pip-pip-egg-info-wgcvwq6w 2025-01-17T10:46:38,783 Preparing metadata (setup.py): started 2025-01-17T10:46:38,785 Running command python setup.py egg_info 2025-01-17T10:46:39,277 WARNING:root:target file does not exist: environments/minimal_requires.txt 2025-01-17T10:46:39,278 WARNING:root:target file does not exist: environments/science_requires.txt 2025-01-17T10:46:39,278 WARNING:root:target file does not exist: environments/dist_requires.txt 2025-01-17T10:46:39,279 WARNING:root:target file does not exist: environments/dev_requires.txt 2025-01-17T10:46:39,280 WARNING:root:target file does not exist: environments/preprocess_requires.txt 2025-01-17T10:46:39,281 WARNING:root:target file does not exist: environments/quality_classifier_requires.txt 2025-01-17T10:46:39,281 WARNING:root:target file does not exist: environments/sandbox_requires.txt 2025-01-17T10:46:39,884 INFO:root:running egg_info 2025-01-17T10:46:39,917 INFO:root:creating /tmp/pip-pip-egg-info-wgcvwq6w/py_data_juicer.egg-info 2025-01-17T10:46:39,920 INFO:root:writing /tmp/pip-pip-egg-info-wgcvwq6w/py_data_juicer.egg-info/PKG-INFO 2025-01-17T10:46:39,941 INFO:root:writing dependency_links to /tmp/pip-pip-egg-info-wgcvwq6w/py_data_juicer.egg-info/dependency_links.txt 2025-01-17T10:46:39,949 INFO:root:writing entry points to /tmp/pip-pip-egg-info-wgcvwq6w/py_data_juicer.egg-info/entry_points.txt 2025-01-17T10:46:39,954 INFO:root:writing requirements to /tmp/pip-pip-egg-info-wgcvwq6w/py_data_juicer.egg-info/requires.txt 2025-01-17T10:46:39,957 INFO:root:writing top-level names to /tmp/pip-pip-egg-info-wgcvwq6w/py_data_juicer.egg-info/top_level.txt 2025-01-17T10:46:39,963 INFO:root:writing manifest file '/tmp/pip-pip-egg-info-wgcvwq6w/py_data_juicer.egg-info/SOURCES.txt' 2025-01-17T10:46:40,081 INFO:root:reading manifest file '/tmp/pip-pip-egg-info-wgcvwq6w/py_data_juicer.egg-info/SOURCES.txt' 2025-01-17T10:46:40,082 INFO:root:adding license file 'LICENSE' 2025-01-17T10:46:40,091 INFO:root:writing manifest file '/tmp/pip-pip-egg-info-wgcvwq6w/py_data_juicer.egg-info/SOURCES.txt' 2025-01-17T10:46:40,186 Preparing metadata (setup.py): finished with status 'done' 2025-01-17T10:46:40,193 Source in /tmp/pip-wheel-fz4yxowk/py-data-juicer_5922be90fbb54d05a6b7fced05bd0658 has version 1.1.0, which satisfies requirement py-data-juicer==1.1.0 from https://files.pythonhosted.org/packages/69/43/f20a50fdbfc0ba44a4816d3c7f9bfcfaf55682ffebf78a10eeff6b40feeb/py_data_juicer-1.1.0.tar.gz 2025-01-17T10:46:40,194 Removed py-data-juicer==1.1.0 from https://files.pythonhosted.org/packages/69/43/f20a50fdbfc0ba44a4816d3c7f9bfcfaf55682ffebf78a10eeff6b40feeb/py_data_juicer-1.1.0.tar.gz from build tracker '/tmp/pip-build-tracker-9kl3el4x' 2025-01-17T10:46:40,202 Created temporary directory: /tmp/pip-unpack-i82dwx38 2025-01-17T10:46:40,204 Created temporary directory: /tmp/pip-unpack-6k3dcl9k 2025-01-17T10:46:40,412 Building wheels for collected packages: py-data-juicer 2025-01-17T10:46:40,417 Created temporary directory: /tmp/pip-wheel-pj_hir46 2025-01-17T10:46:40,418 Building wheel for py-data-juicer (setup.py): started 2025-01-17T10:46:40,419 Destination directory: /tmp/pip-wheel-pj_hir46 2025-01-17T10:46:40,420 Running command python setup.py bdist_wheel 2025-01-17T10:46:41,124 WARNING:root:target file does not exist: environments/minimal_requires.txt 2025-01-17T10:46:41,125 WARNING:root:target file does not exist: environments/science_requires.txt 2025-01-17T10:46:41,126 WARNING:root:target file does not exist: environments/dist_requires.txt 2025-01-17T10:46:41,127 WARNING:root:target file does not exist: environments/dev_requires.txt 2025-01-17T10:46:41,128 WARNING:root:target file does not exist: environments/preprocess_requires.txt 2025-01-17T10:46:41,129 WARNING:root:target file does not exist: environments/quality_classifier_requires.txt 2025-01-17T10:46:41,130 WARNING:root:target file does not exist: environments/sandbox_requires.txt 2025-01-17T10:46:41,757 INFO:root:running bdist_wheel 2025-01-17T10:46:41,890 INFO:root:running build 2025-01-17T10:46:41,891 INFO:root:running build_py 2025-01-17T10:46:41,921 INFO:root:creating build/lib/data_juicer 2025-01-17T10:46:41,923 INFO:root:copying data_juicer/__init__.py -> build/lib/data_juicer 2025-01-17T10:46:41,925 INFO:root:creating build/lib/data_juicer/config 2025-01-17T10:46:41,926 INFO:root:copying data_juicer/config/config.py -> build/lib/data_juicer/config 2025-01-17T10:46:41,930 INFO:root:copying data_juicer/config/__init__.py -> build/lib/data_juicer/config 2025-01-17T10:46:41,933 INFO:root:creating build/lib/data_juicer/utils 2025-01-17T10:46:41,934 INFO:root:copying data_juicer/utils/process_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,936 INFO:root:copying data_juicer/utils/common_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,938 INFO:root:copying data_juicer/utils/fingerprint_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,940 INFO:root:copying data_juicer/utils/availability_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,942 INFO:root:copying data_juicer/utils/lazy_loader.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,944 INFO:root:copying data_juicer/utils/file_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,946 INFO:root:copying data_juicer/utils/resource_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,948 INFO:root:copying data_juicer/utils/constant.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,951 INFO:root:copying data_juicer/utils/registry.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,953 INFO:root:copying data_juicer/utils/cache_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,955 INFO:root:copying data_juicer/utils/logger_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,957 INFO:root:copying data_juicer/utils/unittest_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,959 INFO:root:copying data_juicer/utils/compress.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,962 INFO:root:copying data_juicer/utils/auto_install_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,964 INFO:root:copying data_juicer/utils/model_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,967 INFO:root:copying data_juicer/utils/mm_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,970 INFO:root:copying data_juicer/utils/auto_install_mapping.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,972 INFO:root:copying data_juicer/utils/asset_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,974 INFO:root:copying data_juicer/utils/__init__.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,975 INFO:root:copying data_juicer/utils/ckpt_utils.py -> build/lib/data_juicer/utils 2025-01-17T10:46:41,978 INFO:root:creating build/lib/data_juicer/format 2025-01-17T10:46:41,979 INFO:root:copying data_juicer/format/mixture_formatter.py -> build/lib/data_juicer/format 2025-01-17T10:46:41,982 INFO:root:copying data_juicer/format/json_formatter.py -> build/lib/data_juicer/format 2025-01-17T10:46:41,984 INFO:root:copying data_juicer/format/empty_formatter.py -> build/lib/data_juicer/format 2025-01-17T10:46:41,986 INFO:root:copying data_juicer/format/csv_formatter.py -> build/lib/data_juicer/format 2025-01-17T10:46:41,988 INFO:root:copying data_juicer/format/parquet_formatter.py -> build/lib/data_juicer/format 2025-01-17T10:46:41,989 INFO:root:copying data_juicer/format/tsv_formatter.py -> build/lib/data_juicer/format 2025-01-17T10:46:41,991 INFO:root:copying data_juicer/format/__init__.py -> build/lib/data_juicer/format 2025-01-17T10:46:41,993 INFO:root:copying data_juicer/format/load.py -> build/lib/data_juicer/format 2025-01-17T10:46:41,995 INFO:root:copying data_juicer/format/text_formatter.py -> build/lib/data_juicer/format 2025-01-17T10:46:41,997 INFO:root:copying data_juicer/format/formatter.py -> build/lib/data_juicer/format 2025-01-17T10:46:42,001 INFO:root:creating build/lib/data_juicer/analysis 2025-01-17T10:46:42,002 INFO:root:copying data_juicer/analysis/diversity_analysis.py -> build/lib/data_juicer/analysis 2025-01-17T10:46:42,004 INFO:root:copying data_juicer/analysis/column_wise_analysis.py -> build/lib/data_juicer/analysis 2025-01-17T10:46:42,007 INFO:root:copying data_juicer/analysis/draw.py -> build/lib/data_juicer/analysis 2025-01-17T10:46:42,009 INFO:root:copying data_juicer/analysis/collector.py -> build/lib/data_juicer/analysis 2025-01-17T10:46:42,010 INFO:root:copying data_juicer/analysis/measure.py -> build/lib/data_juicer/analysis 2025-01-17T10:46:42,013 INFO:root:copying data_juicer/analysis/__init__.py -> build/lib/data_juicer/analysis 2025-01-17T10:46:42,014 INFO:root:copying data_juicer/analysis/overall_analysis.py -> build/lib/data_juicer/analysis 2025-01-17T10:46:42,017 INFO:root:creating build/lib/data_juicer/ops 2025-01-17T10:46:42,018 INFO:root:copying data_juicer/ops/op_fusion.py -> build/lib/data_juicer/ops 2025-01-17T10:46:42,021 INFO:root:copying data_juicer/ops/base_op.py -> build/lib/data_juicer/ops 2025-01-17T10:46:42,023 INFO:root:copying data_juicer/ops/__init__.py -> build/lib/data_juicer/ops 2025-01-17T10:46:42,025 INFO:root:copying data_juicer/ops/load.py -> build/lib/data_juicer/ops 2025-01-17T10:46:42,027 INFO:root:creating build/lib/data_juicer/core 2025-01-17T10:46:42,028 INFO:root:copying data_juicer/core/adapter.py -> build/lib/data_juicer/core 2025-01-17T10:46:42,031 INFO:root:copying data_juicer/core/monitor.py -> build/lib/data_juicer/core 2025-01-17T10:46:42,033 INFO:root:copying data_juicer/core/tracer.py -> build/lib/data_juicer/core 2025-01-17T10:46:42,036 INFO:root:copying data_juicer/core/analyzer.py -> build/lib/data_juicer/core 2025-01-17T10:46:42,038 INFO:root:copying data_juicer/core/ray_data.py -> build/lib/data_juicer/core 2025-01-17T10:46:42,041 INFO:root:copying data_juicer/core/executor.py -> build/lib/data_juicer/core 2025-01-17T10:46:42,043 INFO:root:copying data_juicer/core/data.py -> build/lib/data_juicer/core 2025-01-17T10:46:42,046 INFO:root:copying data_juicer/core/__init__.py -> build/lib/data_juicer/core 2025-01-17T10:46:42,048 INFO:root:copying data_juicer/core/exporter.py -> build/lib/data_juicer/core 2025-01-17T10:46:42,051 INFO:root:copying data_juicer/core/ray_executor.py -> build/lib/data_juicer/core 2025-01-17T10:46:42,054 INFO:root:creating build/lib/data_juicer/ops/selector 2025-01-17T10:46:42,055 INFO:root:copying data_juicer/ops/selector/topk_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-01-17T10:46:42,057 INFO:root:copying data_juicer/ops/selector/range_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-01-17T10:46:42,059 INFO:root:copying data_juicer/ops/selector/frequency_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-01-17T10:46:42,061 INFO:root:copying data_juicer/ops/selector/random_selector.py -> build/lib/data_juicer/ops/selector 2025-01-17T10:46:42,063 INFO:root:copying data_juicer/ops/selector/tags_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-01-17T10:46:42,065 INFO:root:copying data_juicer/ops/selector/__init__.py -> build/lib/data_juicer/ops/selector 2025-01-17T10:46:42,067 INFO:root:creating build/lib/data_juicer/ops/grouper 2025-01-17T10:46:42,068 INFO:root:copying data_juicer/ops/grouper/naive_reverse_grouper.py -> build/lib/data_juicer/ops/grouper 2025-01-17T10:46:42,071 INFO:root:copying data_juicer/ops/grouper/key_value_grouper.py -> build/lib/data_juicer/ops/grouper 2025-01-17T10:46:42,073 INFO:root:copying data_juicer/ops/grouper/naive_grouper.py -> build/lib/data_juicer/ops/grouper 2025-01-17T10:46:42,074 INFO:root:copying data_juicer/ops/grouper/__init__.py -> build/lib/data_juicer/ops/grouper 2025-01-17T10:46:42,077 INFO:root:creating build/lib/data_juicer/ops/aggregator 2025-01-17T10:46:42,078 INFO:root:copying data_juicer/ops/aggregator/nested_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-01-17T10:46:42,080 INFO:root:copying data_juicer/ops/aggregator/most_relavant_entities_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-01-17T10:46:42,082 INFO:root:copying data_juicer/ops/aggregator/meta_tags_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-01-17T10:46:42,085 INFO:root:copying data_juicer/ops/aggregator/entity_attribute_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-01-17T10:46:42,087 INFO:root:copying data_juicer/ops/aggregator/__init__.py -> build/lib/data_juicer/ops/aggregator 2025-01-17T10:46:42,091 INFO:root:creating build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,093 INFO:root:copying data_juicer/ops/filter/word_repetition_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,096 INFO:root:copying data_juicer/ops/filter/image_shape_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,098 INFO:root:copying data_juicer/ops/filter/stopwords_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,100 INFO:root:copying data_juicer/ops/filter/image_face_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,102 INFO:root:copying data_juicer/ops/filter/phrase_grounding_recall_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,105 INFO:root:copying data_juicer/ops/filter/maximum_line_length_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,106 INFO:root:copying data_juicer/ops/filter/image_watermark_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,108 INFO:root:copying data_juicer/ops/filter/video_motion_score_raft_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,110 INFO:root:copying data_juicer/ops/filter/alphanumeric_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,112 INFO:root:copying data_juicer/ops/filter/text_entity_dependency_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,114 INFO:root:copying data_juicer/ops/filter/suffix_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,116 INFO:root:copying data_juicer/ops/filter/video_aesthetics_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,118 INFO:root:copying data_juicer/ops/filter/specified_numeric_field_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,120 INFO:root:copying data_juicer/ops/filter/image_text_matching_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,122 INFO:root:copying data_juicer/ops/filter/image_face_count_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,124 INFO:root:copying data_juicer/ops/filter/words_num_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,126 INFO:root:copying data_juicer/ops/filter/perplexity_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,128 INFO:root:copying data_juicer/ops/filter/audio_nmf_snr_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,130 INFO:root:copying data_juicer/ops/filter/image_pair_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,133 INFO:root:copying data_juicer/ops/filter/image_aesthetics_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,137 INFO:root:copying data_juicer/ops/filter/specified_field_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,139 INFO:root:copying data_juicer/ops/filter/image_text_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,142 INFO:root:copying data_juicer/ops/filter/image_nsfw_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,144 INFO:root:copying data_juicer/ops/filter/text_pair_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,147 INFO:root:copying data_juicer/ops/filter/token_num_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,149 INFO:root:copying data_juicer/ops/filter/video_tagging_from_frames_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,151 INFO:root:copying data_juicer/ops/filter/text_length_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,154 INFO:root:copying data_juicer/ops/filter/text_action_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,156 INFO:root:copying data_juicer/ops/filter/audio_duration_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,158 INFO:root:copying data_juicer/ops/filter/special_characters_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,161 INFO:root:copying data_juicer/ops/filter/video_nsfw_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,163 INFO:root:copying data_juicer/ops/filter/audio_size_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,165 INFO:root:copying data_juicer/ops/filter/video_motion_score_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,168 INFO:root:copying data_juicer/ops/filter/image_size_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,170 INFO:root:copying data_juicer/ops/filter/average_line_length_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,173 INFO:root:copying data_juicer/ops/filter/video_watermark_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,175 INFO:root:copying data_juicer/ops/filter/__init__.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,177 INFO:root:copying data_juicer/ops/filter/video_ocr_area_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,180 INFO:root:copying data_juicer/ops/filter/flagged_words_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,183 INFO:root:copying data_juicer/ops/filter/image_aspect_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,186 INFO:root:copying data_juicer/ops/filter/language_id_score_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,190 INFO:root:copying data_juicer/ops/filter/video_duration_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,192 INFO:root:copying data_juicer/ops/filter/character_repetition_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,194 INFO:root:copying data_juicer/ops/filter/video_aspect_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,196 INFO:root:copying data_juicer/ops/filter/video_resolution_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,198 INFO:root:copying data_juicer/ops/filter/video_frames_text_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-01-17T10:46:42,202 INFO:root:creating build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,204 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,208 INFO:root:copying data_juicer/ops/mapper/video_tagging_from_audio_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,212 INFO:root:copying data_juicer/ops/mapper/query_topic_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,216 INFO:root:copying data_juicer/ops/mapper/optimize_response_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,218 INFO:root:copying data_juicer/ops/mapper/text_chunk_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,222 INFO:root:copying data_juicer/ops/mapper/extract_keyword_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,225 INFO:root:copying data_juicer/ops/mapper/calibrate_response_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,233 INFO:root:copying data_juicer/ops/mapper/video_split_by_duration_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,240 INFO:root:copying data_juicer/ops/mapper/python_file_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,245 INFO:root:copying data_juicer/ops/mapper/punctuation_normalization_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,249 INFO:root:copying data_juicer/ops/mapper/image_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,255 INFO:root:copying data_juicer/ops/mapper/pair_preference_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,259 INFO:root:copying data_juicer/ops/mapper/sentence_split_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,262 INFO:root:copying data_juicer/ops/mapper/dialog_sentiment_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,265 INFO:root:copying data_juicer/ops/mapper/query_intent_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,268 INFO:root:copying data_juicer/ops/mapper/fix_unicode_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,271 INFO:root:copying data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,281 INFO:root:copying data_juicer/ops/mapper/remove_non_chinese_character_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,296 INFO:root:copying data_juicer/ops/mapper/video_resize_resolution_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,301 INFO:root:copying data_juicer/ops/mapper/remove_header_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,303 INFO:root:copying data_juicer/ops/mapper/calibrate_query_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,305 INFO:root:copying data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,308 INFO:root:copying data_juicer/ops/mapper/extract_nickname_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,311 INFO:root:copying data_juicer/ops/mapper/query_sentiment_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,316 INFO:root:copying data_juicer/ops/mapper/dialog_sentiment_intensity_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,320 INFO:root:copying data_juicer/ops/mapper/extract_support_text_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,324 INFO:root:copying data_juicer/ops/mapper/remove_bibliography_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,329 INFO:root:copying data_juicer/ops/mapper/whitespace_normalization_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,334 INFO:root:copying data_juicer/ops/mapper/video_tagging_from_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,338 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_video_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,549 INFO:root:copying data_juicer/ops/mapper/generate_qa_from_examples_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,551 INFO:root:copying data_juicer/ops/mapper/extract_entity_relation_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,555 INFO:root:copying data_juicer/ops/mapper/sdxl_prompt2prompt_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,559 INFO:root:copying data_juicer/ops/mapper/image_tagging_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,561 INFO:root:copying data_juicer/ops/mapper/image_captioning_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,564 INFO:root:copying data_juicer/ops/mapper/image_face_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,567 INFO:root:copying data_juicer/ops/mapper/dialog_intent_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,569 INFO:root:copying data_juicer/ops/mapper/video_extract_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,572 INFO:root:copying data_juicer/ops/mapper/extract_event_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,574 INFO:root:copying data_juicer/ops/mapper/python_lambda_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,577 INFO:root:copying data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,579 INFO:root:copying data_juicer/ops/mapper/clean_links_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,581 INFO:root:copying data_juicer/ops/mapper/calibrate_qa_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,583 INFO:root:copying data_juicer/ops/mapper/remove_repeat_sentences_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,586 INFO:root:copying data_juicer/ops/mapper/remove_table_text_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,590 INFO:root:copying data_juicer/ops/mapper/extract_entity_attribute_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,595 INFO:root:copying data_juicer/ops/mapper/clean_ip_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,601 INFO:root:copying data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,604 INFO:root:copying data_juicer/ops/mapper/nlpaug_en_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,608 INFO:root:copying data_juicer/ops/mapper/chinese_convert_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,610 INFO:root:copying data_juicer/ops/mapper/video_split_by_scene_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,613 INFO:root:copying data_juicer/ops/mapper/remove_specific_chars_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,615 INFO:root:copying data_juicer/ops/mapper/nlpcda_zh_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,618 INFO:root:copying data_juicer/ops/mapper/relation_identity_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,621 INFO:root:copying data_juicer/ops/mapper/remove_comments_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,623 INFO:root:copying data_juicer/ops/mapper/replace_content_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,625 INFO:root:copying data_juicer/ops/mapper/clean_copyright_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,628 INFO:root:copying data_juicer/ops/mapper/expand_macro_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,630 INFO:root:copying data_juicer/ops/mapper/optimize_query_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,633 INFO:root:copying data_juicer/ops/mapper/mllm_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,637 INFO:root:copying data_juicer/ops/mapper/clean_html_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,639 INFO:root:copying data_juicer/ops/mapper/video_remove_watermark_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,643 INFO:root:copying data_juicer/ops/mapper/__init__.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,646 INFO:root:copying data_juicer/ops/mapper/video_split_by_key_frame_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,649 INFO:root:copying data_juicer/ops/mapper/optimize_qa_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,652 INFO:root:copying data_juicer/ops/mapper/video_face_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,655 INFO:root:copying data_juicer/ops/mapper/dialog_topic_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,658 INFO:root:copying data_juicer/ops/mapper/remove_long_words_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,661 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,664 INFO:root:copying data_juicer/ops/mapper/clean_email_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,666 INFO:root:copying data_juicer/ops/mapper/image_diffusion_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,669 INFO:root:copying data_juicer/ops/mapper/generate_qa_from_text_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,672 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_audio_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,676 INFO:root:copying data_juicer/ops/mapper/image_segment_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,679 INFO:root:copying data_juicer/ops/mapper/sentence_augmentation_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,681 INFO:root:copying data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py -> build/lib/data_juicer/ops/mapper 2025-01-17T10:46:42,685 INFO:root:creating build/lib/data_juicer/ops/common 2025-01-17T10:46:42,687 INFO:root:copying data_juicer/ops/common/special_characters.py -> build/lib/data_juicer/ops/common 2025-01-17T10:46:42,689 INFO:root:copying data_juicer/ops/common/__init__.py -> build/lib/data_juicer/ops/common 2025-01-17T10:46:42,691 INFO:root:copying data_juicer/ops/common/helper_func.py -> build/lib/data_juicer/ops/common 2025-01-17T10:46:42,694 INFO:root:creating build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,695 INFO:root:copying data_juicer/ops/deduplicator/ray_document_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,698 INFO:root:copying data_juicer/ops/deduplicator/ray_bts_minhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,701 INFO:root:copying data_juicer/ops/deduplicator/ray_image_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,703 INFO:root:copying data_juicer/ops/deduplicator/document_simhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,705 INFO:root:copying data_juicer/ops/deduplicator/ray_basic_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,707 INFO:root:copying data_juicer/ops/deduplicator/document_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,709 INFO:root:copying data_juicer/ops/deduplicator/video_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,712 INFO:root:copying data_juicer/ops/deduplicator/__init__.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,714 INFO:root:copying data_juicer/ops/deduplicator/image_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,717 INFO:root:copying data_juicer/ops/deduplicator/document_minhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,720 INFO:root:copying data_juicer/ops/deduplicator/ray_video_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-01-17T10:46:42,723 INFO:root:creating build/lib/data_juicer/tools 2025-01-17T10:46:42,723 INFO:root:copying tools/analyze_data.py -> build/lib/data_juicer/tools 2025-01-17T10:46:42,726 INFO:root:copying tools/sandbox_starter.py -> build/lib/data_juicer/tools 2025-01-17T10:46:42,727 INFO:root:copying tools/process_data.py -> build/lib/data_juicer/tools 2025-01-17T10:46:42,729 INFO:root:copying tools/dj_install.py -> build/lib/data_juicer/tools 2025-01-17T10:46:42,731 INFO:root:copying tools/__init__.py -> build/lib/data_juicer/tools 2025-01-17T10:46:42,784 /usr/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. 2025-01-17T10:46:42,784 !! 2025-01-17T10:46:42,785 ******************************************************************************** 2025-01-17T10:46:42,786 Please avoid running ``setup.py`` directly. 2025-01-17T10:46:42,787 Instead, use pypa/build, pypa/installer or other 2025-01-17T10:46:42,787 standards-based tools. 2025-01-17T10:46:42,789 See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. 2025-01-17T10:46:42,789 ******************************************************************************** 2025-01-17T10:46:42,790 !! 2025-01-17T10:46:42,791 self.initialize_options() 2025-01-17T10:46:42,812 INFO:root:installing to build/bdist.linux-armv7l/wheel 2025-01-17T10:46:42,813 INFO:root:running install 2025-01-17T10:46:42,837 INFO:root:running install_lib 2025-01-17T10:46:42,875 INFO:root:creating build/bdist.linux-armv7l/wheel 2025-01-17T10:46:42,878 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer 2025-01-17T10:46:42,881 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/config 2025-01-17T10:46:42,882 INFO:root:copying build/lib/data_juicer/config/config.py -> build/bdist.linux-armv7l/wheel/./data_juicer/config 2025-01-17T10:46:42,886 INFO:root:copying build/lib/data_juicer/config/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/config 2025-01-17T10:46:42,890 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/tools 2025-01-17T10:46:42,891 INFO:root:copying build/lib/data_juicer/tools/analyze_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-01-17T10:46:42,894 INFO:root:copying build/lib/data_juicer/tools/sandbox_starter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-01-17T10:46:42,896 INFO:root:copying build/lib/data_juicer/tools/process_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-01-17T10:46:42,898 INFO:root:copying build/lib/data_juicer/tools/dj_install.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-01-17T10:46:42,900 INFO:root:copying build/lib/data_juicer/tools/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-01-17T10:46:42,903 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/utils 2025-01-17T10:46:42,904 INFO:root:copying build/lib/data_juicer/utils/process_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,907 INFO:root:copying build/lib/data_juicer/utils/common_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,909 INFO:root:copying build/lib/data_juicer/utils/fingerprint_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,912 INFO:root:copying build/lib/data_juicer/utils/availability_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,914 INFO:root:copying build/lib/data_juicer/utils/lazy_loader.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,916 INFO:root:copying build/lib/data_juicer/utils/file_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,919 INFO:root:copying build/lib/data_juicer/utils/resource_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,921 INFO:root:copying build/lib/data_juicer/utils/constant.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,924 INFO:root:copying build/lib/data_juicer/utils/registry.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,927 INFO:root:copying build/lib/data_juicer/utils/cache_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,929 INFO:root:copying build/lib/data_juicer/utils/logger_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,932 INFO:root:copying build/lib/data_juicer/utils/unittest_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,935 INFO:root:copying build/lib/data_juicer/utils/compress.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,938 INFO:root:copying build/lib/data_juicer/utils/auto_install_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,941 INFO:root:copying build/lib/data_juicer/utils/model_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,945 INFO:root:copying build/lib/data_juicer/utils/mm_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,948 INFO:root:copying build/lib/data_juicer/utils/auto_install_mapping.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,950 INFO:root:copying build/lib/data_juicer/utils/asset_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,952 INFO:root:copying build/lib/data_juicer/utils/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,954 INFO:root:copying build/lib/data_juicer/utils/ckpt_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-01-17T10:46:42,957 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/format 2025-01-17T10:46:42,958 INFO:root:copying build/lib/data_juicer/format/mixture_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-17T10:46:42,960 INFO:root:copying build/lib/data_juicer/format/json_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-17T10:46:42,962 INFO:root:copying build/lib/data_juicer/format/empty_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-17T10:46:42,964 INFO:root:copying build/lib/data_juicer/format/csv_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-17T10:46:42,966 INFO:root:copying build/lib/data_juicer/format/parquet_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-17T10:46:42,967 INFO:root:copying build/lib/data_juicer/format/tsv_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-17T10:46:42,969 INFO:root:copying build/lib/data_juicer/format/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-17T10:46:42,971 INFO:root:copying build/lib/data_juicer/format/load.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-17T10:46:42,972 INFO:root:copying build/lib/data_juicer/format/text_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-17T10:46:42,975 INFO:root:copying build/lib/data_juicer/format/formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-01-17T10:46:42,978 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/analysis 2025-01-17T10:46:42,979 INFO:root:copying build/lib/data_juicer/analysis/diversity_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-17T10:46:42,981 INFO:root:copying build/lib/data_juicer/analysis/column_wise_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-17T10:46:42,984 INFO:root:copying build/lib/data_juicer/analysis/draw.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-17T10:46:42,986 INFO:root:copying build/lib/data_juicer/analysis/collector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-17T10:46:42,988 INFO:root:copying build/lib/data_juicer/analysis/measure.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-17T10:46:42,990 INFO:root:copying build/lib/data_juicer/analysis/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-17T10:46:42,992 INFO:root:copying build/lib/data_juicer/analysis/overall_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-01-17T10:46:42,994 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops 2025-01-17T10:46:42,997 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/selector 2025-01-17T10:46:42,998 INFO:root:copying build/lib/data_juicer/ops/selector/topk_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-17T10:46:43,000 INFO:root:copying build/lib/data_juicer/ops/selector/range_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-17T10:46:43,002 INFO:root:copying build/lib/data_juicer/ops/selector/frequency_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-17T10:46:43,004 INFO:root:copying build/lib/data_juicer/ops/selector/random_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-17T10:46:43,006 INFO:root:copying build/lib/data_juicer/ops/selector/tags_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-17T10:46:43,008 INFO:root:copying build/lib/data_juicer/ops/selector/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-01-17T10:46:43,010 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/grouper 2025-01-17T10:46:43,011 INFO:root:copying build/lib/data_juicer/ops/grouper/naive_reverse_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-01-17T10:46:43,014 INFO:root:copying build/lib/data_juicer/ops/grouper/key_value_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-01-17T10:46:43,016 INFO:root:copying build/lib/data_juicer/ops/grouper/naive_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-01-17T10:46:43,018 INFO:root:copying build/lib/data_juicer/ops/grouper/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-01-17T10:46:43,019 INFO:root:copying build/lib/data_juicer/ops/op_fusion.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-01-17T10:46:43,022 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/aggregator 2025-01-17T10:46:43,024 INFO:root:copying build/lib/data_juicer/ops/aggregator/nested_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-01-17T10:46:43,026 INFO:root:copying build/lib/data_juicer/ops/aggregator/most_relavant_entities_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-01-17T10:46:43,029 INFO:root:copying build/lib/data_juicer/ops/aggregator/meta_tags_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-01-17T10:46:43,031 INFO:root:copying build/lib/data_juicer/ops/aggregator/entity_attribute_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-01-17T10:46:43,033 INFO:root:copying build/lib/data_juicer/ops/aggregator/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-01-17T10:46:43,036 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/filter 2025-01-17T10:46:43,037 INFO:root:copying build/lib/data_juicer/ops/filter/word_repetition_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,040 INFO:root:copying build/lib/data_juicer/ops/filter/image_shape_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,042 INFO:root:copying build/lib/data_juicer/ops/filter/stopwords_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,044 INFO:root:copying build/lib/data_juicer/ops/filter/image_face_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,046 INFO:root:copying build/lib/data_juicer/ops/filter/phrase_grounding_recall_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,049 INFO:root:copying build/lib/data_juicer/ops/filter/maximum_line_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,050 INFO:root:copying build/lib/data_juicer/ops/filter/image_watermark_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,052 INFO:root:copying build/lib/data_juicer/ops/filter/video_motion_score_raft_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,054 INFO:root:copying build/lib/data_juicer/ops/filter/alphanumeric_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,056 INFO:root:copying build/lib/data_juicer/ops/filter/text_entity_dependency_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,058 INFO:root:copying build/lib/data_juicer/ops/filter/suffix_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,059 INFO:root:copying build/lib/data_juicer/ops/filter/video_aesthetics_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,062 INFO:root:copying build/lib/data_juicer/ops/filter/specified_numeric_field_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,064 INFO:root:copying build/lib/data_juicer/ops/filter/image_text_matching_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,067 INFO:root:copying build/lib/data_juicer/ops/filter/image_face_count_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,069 INFO:root:copying build/lib/data_juicer/ops/filter/words_num_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,071 INFO:root:copying build/lib/data_juicer/ops/filter/perplexity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,073 INFO:root:copying build/lib/data_juicer/ops/filter/audio_nmf_snr_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,075 INFO:root:copying build/lib/data_juicer/ops/filter/image_pair_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,077 INFO:root:copying build/lib/data_juicer/ops/filter/image_aesthetics_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,079 INFO:root:copying build/lib/data_juicer/ops/filter/specified_field_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,082 INFO:root:copying build/lib/data_juicer/ops/filter/image_text_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,084 INFO:root:copying build/lib/data_juicer/ops/filter/image_nsfw_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,086 INFO:root:copying build/lib/data_juicer/ops/filter/text_pair_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,088 INFO:root:copying build/lib/data_juicer/ops/filter/token_num_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,090 INFO:root:copying build/lib/data_juicer/ops/filter/video_tagging_from_frames_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,092 INFO:root:copying build/lib/data_juicer/ops/filter/text_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,094 INFO:root:copying build/lib/data_juicer/ops/filter/text_action_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,096 INFO:root:copying build/lib/data_juicer/ops/filter/audio_duration_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,098 INFO:root:copying build/lib/data_juicer/ops/filter/special_characters_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,099 INFO:root:copying build/lib/data_juicer/ops/filter/video_nsfw_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,102 INFO:root:copying build/lib/data_juicer/ops/filter/audio_size_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,103 INFO:root:copying build/lib/data_juicer/ops/filter/video_motion_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,106 INFO:root:copying build/lib/data_juicer/ops/filter/image_size_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,107 INFO:root:copying build/lib/data_juicer/ops/filter/average_line_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,109 INFO:root:copying build/lib/data_juicer/ops/filter/video_watermark_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,111 INFO:root:copying build/lib/data_juicer/ops/filter/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,113 INFO:root:copying build/lib/data_juicer/ops/filter/video_ocr_area_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,115 INFO:root:copying build/lib/data_juicer/ops/filter/flagged_words_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,118 INFO:root:copying build/lib/data_juicer/ops/filter/image_aspect_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,119 INFO:root:copying build/lib/data_juicer/ops/filter/language_id_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,121 INFO:root:copying build/lib/data_juicer/ops/filter/video_duration_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,123 INFO:root:copying build/lib/data_juicer/ops/filter/character_repetition_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,125 INFO:root:copying build/lib/data_juicer/ops/filter/video_aspect_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,127 INFO:root:copying build/lib/data_juicer/ops/filter/video_resolution_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,129 INFO:root:copying build/lib/data_juicer/ops/filter/video_frames_text_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-01-17T10:46:43,133 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/mapper 2025-01-17T10:46:43,134 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,136 INFO:root:copying build/lib/data_juicer/ops/mapper/video_tagging_from_audio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,138 INFO:root:copying build/lib/data_juicer/ops/mapper/query_topic_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,140 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_response_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,141 INFO:root:copying build/lib/data_juicer/ops/mapper/text_chunk_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,144 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_keyword_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,146 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_response_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,147 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_duration_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,150 INFO:root:copying build/lib/data_juicer/ops/mapper/python_file_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,151 INFO:root:copying build/lib/data_juicer/ops/mapper/punctuation_normalization_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,153 INFO:root:copying build/lib/data_juicer/ops/mapper/image_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,155 INFO:root:copying build/lib/data_juicer/ops/mapper/pair_preference_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,158 INFO:root:copying build/lib/data_juicer/ops/mapper/sentence_split_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,159 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_sentiment_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,162 INFO:root:copying build/lib/data_juicer/ops/mapper/query_intent_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,163 INFO:root:copying build/lib/data_juicer/ops/mapper/fix_unicode_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,165 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,167 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_non_chinese_character_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,169 INFO:root:copying build/lib/data_juicer/ops/mapper/video_resize_resolution_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,171 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_header_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,173 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_query_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,175 INFO:root:copying build/lib/data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,177 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_nickname_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,179 INFO:root:copying build/lib/data_juicer/ops/mapper/query_sentiment_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,181 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_sentiment_intensity_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,183 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_support_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,185 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_bibliography_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,187 INFO:root:copying build/lib/data_juicer/ops/mapper/whitespace_normalization_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,188 INFO:root:copying build/lib/data_juicer/ops/mapper/video_tagging_from_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,190 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_video_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,193 INFO:root:copying build/lib/data_juicer/ops/mapper/generate_qa_from_examples_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,195 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_entity_relation_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,198 INFO:root:copying build/lib/data_juicer/ops/mapper/sdxl_prompt2prompt_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,202 INFO:root:copying build/lib/data_juicer/ops/mapper/image_tagging_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,204 INFO:root:copying build/lib/data_juicer/ops/mapper/image_captioning_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,206 INFO:root:copying build/lib/data_juicer/ops/mapper/image_face_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,208 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_intent_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,210 INFO:root:copying build/lib/data_juicer/ops/mapper/video_extract_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,213 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_event_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,215 INFO:root:copying build/lib/data_juicer/ops/mapper/python_lambda_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,217 INFO:root:copying build/lib/data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,219 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_links_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,221 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_qa_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,223 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_repeat_sentences_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,225 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_table_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,227 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_entity_attribute_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,229 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_ip_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,231 INFO:root:copying build/lib/data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,234 INFO:root:copying build/lib/data_juicer/ops/mapper/nlpaug_en_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,236 INFO:root:copying build/lib/data_juicer/ops/mapper/chinese_convert_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,238 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_scene_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,240 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_specific_chars_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,242 INFO:root:copying build/lib/data_juicer/ops/mapper/nlpcda_zh_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,244 INFO:root:copying build/lib/data_juicer/ops/mapper/relation_identity_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,246 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_comments_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,248 INFO:root:copying build/lib/data_juicer/ops/mapper/replace_content_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,249 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_copyright_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,251 INFO:root:copying build/lib/data_juicer/ops/mapper/expand_macro_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,253 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_query_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,255 INFO:root:copying build/lib/data_juicer/ops/mapper/mllm_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,257 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_html_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,259 INFO:root:copying build/lib/data_juicer/ops/mapper/video_remove_watermark_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,261 INFO:root:copying build/lib/data_juicer/ops/mapper/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,263 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_key_frame_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,266 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_qa_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,268 INFO:root:copying build/lib/data_juicer/ops/mapper/video_face_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,270 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_topic_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,273 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_long_words_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,274 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,277 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_email_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,278 INFO:root:copying build/lib/data_juicer/ops/mapper/image_diffusion_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,281 INFO:root:copying build/lib/data_juicer/ops/mapper/generate_qa_from_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,283 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_audio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,285 INFO:root:copying build/lib/data_juicer/ops/mapper/image_segment_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,287 INFO:root:copying build/lib/data_juicer/ops/mapper/sentence_augmentation_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,289 INFO:root:copying build/lib/data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-01-17T10:46:43,291 INFO:root:copying build/lib/data_juicer/ops/base_op.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-01-17T10:46:43,294 INFO:root:copying build/lib/data_juicer/ops/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-01-17T10:46:43,296 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/common 2025-01-17T10:46:43,297 INFO:root:copying build/lib/data_juicer/ops/common/special_characters.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2025-01-17T10:46:43,299 INFO:root:copying build/lib/data_juicer/ops/common/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2025-01-17T10:46:43,301 INFO:root:copying build/lib/data_juicer/ops/common/helper_func.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2025-01-17T10:46:43,304 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/deduplicator 2025-01-17T10:46:43,305 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_document_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,307 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_bts_minhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,309 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_image_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,311 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_simhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,313 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_basic_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,315 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,317 INFO:root:copying build/lib/data_juicer/ops/deduplicator/video_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,319 INFO:root:copying build/lib/data_juicer/ops/deduplicator/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,321 INFO:root:copying build/lib/data_juicer/ops/deduplicator/image_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,323 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_minhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,325 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_video_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-01-17T10:46:43,327 INFO:root:copying build/lib/data_juicer/ops/load.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-01-17T10:46:43,329 INFO:root:copying build/lib/data_juicer/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer 2025-01-17T10:46:43,331 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/core 2025-01-17T10:46:43,333 INFO:root:copying build/lib/data_juicer/core/adapter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-17T10:46:43,335 INFO:root:copying build/lib/data_juicer/core/monitor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-17T10:46:43,337 INFO:root:copying build/lib/data_juicer/core/tracer.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-17T10:46:43,339 INFO:root:copying build/lib/data_juicer/core/analyzer.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-17T10:46:43,341 INFO:root:copying build/lib/data_juicer/core/ray_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-17T10:46:43,343 INFO:root:copying build/lib/data_juicer/core/executor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-17T10:46:43,345 INFO:root:copying build/lib/data_juicer/core/data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-17T10:46:43,348 INFO:root:copying build/lib/data_juicer/core/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-17T10:46:43,349 INFO:root:copying build/lib/data_juicer/core/exporter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-17T10:46:43,351 INFO:root:copying build/lib/data_juicer/core/ray_executor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-01-17T10:46:43,353 INFO:root:running install_egg_info 2025-01-17T10:46:43,385 INFO:root:running egg_info 2025-01-17T10:46:43,412 INFO:root:writing py_data_juicer.egg-info/PKG-INFO 2025-01-17T10:46:43,416 INFO:root:writing dependency_links to py_data_juicer.egg-info/dependency_links.txt 2025-01-17T10:46:43,418 INFO:root:writing entry points to py_data_juicer.egg-info/entry_points.txt 2025-01-17T10:46:43,419 INFO:root:writing requirements to py_data_juicer.egg-info/requires.txt 2025-01-17T10:46:43,420 INFO:root:writing top-level names to py_data_juicer.egg-info/top_level.txt 2025-01-17T10:46:43,469 INFO:root:reading manifest file 'py_data_juicer.egg-info/SOURCES.txt' 2025-01-17T10:46:43,481 INFO:root:adding license file 'LICENSE' 2025-01-17T10:46:43,493 INFO:root:writing manifest file 'py_data_juicer.egg-info/SOURCES.txt' 2025-01-17T10:46:43,495 INFO:root:Copying py_data_juicer.egg-info to build/bdist.linux-armv7l/wheel/./py_data_juicer-1.1.0-py3.11.egg-info 2025-01-17T10:46:43,507 INFO:root:running install_scripts 2025-01-17T10:46:43,526 INFO:root:creating build/bdist.linux-armv7l/wheel/py_data_juicer-1.1.0.dist-info/WHEEL 2025-01-17T10:46:43,528 INFO:wheel:creating '/tmp/pip-wheel-pj_hir46/py_data_juicer-1.1.0-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2025-01-17T10:46:43,530 INFO:wheel:adding 'data_juicer/__init__.py' 2025-01-17T10:46:43,532 INFO:wheel:adding 'data_juicer/analysis/__init__.py' 2025-01-17T10:46:43,533 INFO:wheel:adding 'data_juicer/analysis/collector.py' 2025-01-17T10:46:43,535 INFO:wheel:adding 'data_juicer/analysis/column_wise_analysis.py' 2025-01-17T10:46:43,537 INFO:wheel:adding 'data_juicer/analysis/diversity_analysis.py' 2025-01-17T10:46:43,539 INFO:wheel:adding 'data_juicer/analysis/draw.py' 2025-01-17T10:46:43,540 INFO:wheel:adding 'data_juicer/analysis/measure.py' 2025-01-17T10:46:43,542 INFO:wheel:adding 'data_juicer/analysis/overall_analysis.py' 2025-01-17T10:46:43,544 INFO:wheel:adding 'data_juicer/config/__init__.py' 2025-01-17T10:46:43,548 INFO:wheel:adding 'data_juicer/config/config.py' 2025-01-17T10:46:43,550 INFO:wheel:adding 'data_juicer/core/__init__.py' 2025-01-17T10:46:43,552 INFO:wheel:adding 'data_juicer/core/adapter.py' 2025-01-17T10:46:43,554 INFO:wheel:adding 'data_juicer/core/analyzer.py' 2025-01-17T10:46:43,556 INFO:wheel:adding 'data_juicer/core/data.py' 2025-01-17T10:46:43,558 INFO:wheel:adding 'data_juicer/core/executor.py' 2025-01-17T10:46:43,560 INFO:wheel:adding 'data_juicer/core/exporter.py' 2025-01-17T10:46:43,561 INFO:wheel:adding 'data_juicer/core/monitor.py' 2025-01-17T10:46:43,563 INFO:wheel:adding 'data_juicer/core/ray_data.py' 2025-01-17T10:46:43,565 INFO:wheel:adding 'data_juicer/core/ray_executor.py' 2025-01-17T10:46:43,566 INFO:wheel:adding 'data_juicer/core/tracer.py' 2025-01-17T10:46:43,568 INFO:wheel:adding 'data_juicer/format/__init__.py' 2025-01-17T10:46:43,569 INFO:wheel:adding 'data_juicer/format/csv_formatter.py' 2025-01-17T10:46:43,571 INFO:wheel:adding 'data_juicer/format/empty_formatter.py' 2025-01-17T10:46:43,573 INFO:wheel:adding 'data_juicer/format/formatter.py' 2025-01-17T10:46:43,574 INFO:wheel:adding 'data_juicer/format/json_formatter.py' 2025-01-17T10:46:43,575 INFO:wheel:adding 'data_juicer/format/load.py' 2025-01-17T10:46:43,576 INFO:wheel:adding 'data_juicer/format/mixture_formatter.py' 2025-01-17T10:46:43,578 INFO:wheel:adding 'data_juicer/format/parquet_formatter.py' 2025-01-17T10:46:43,579 INFO:wheel:adding 'data_juicer/format/text_formatter.py' 2025-01-17T10:46:43,580 INFO:wheel:adding 'data_juicer/format/tsv_formatter.py' 2025-01-17T10:46:43,582 INFO:wheel:adding 'data_juicer/ops/__init__.py' 2025-01-17T10:46:43,584 INFO:wheel:adding 'data_juicer/ops/base_op.py' 2025-01-17T10:46:43,586 INFO:wheel:adding 'data_juicer/ops/load.py' 2025-01-17T10:46:43,587 INFO:wheel:adding 'data_juicer/ops/op_fusion.py' 2025-01-17T10:46:43,589 INFO:wheel:adding 'data_juicer/ops/aggregator/__init__.py' 2025-01-17T10:46:43,591 INFO:wheel:adding 'data_juicer/ops/aggregator/entity_attribute_aggregator.py' 2025-01-17T10:46:43,593 INFO:wheel:adding 'data_juicer/ops/aggregator/meta_tags_aggregator.py' 2025-01-17T10:46:43,594 INFO:wheel:adding 'data_juicer/ops/aggregator/most_relavant_entities_aggregator.py' 2025-01-17T10:46:43,596 INFO:wheel:adding 'data_juicer/ops/aggregator/nested_aggregator.py' 2025-01-17T10:46:43,598 INFO:wheel:adding 'data_juicer/ops/common/__init__.py' 2025-01-17T10:46:43,599 INFO:wheel:adding 'data_juicer/ops/common/helper_func.py' 2025-01-17T10:46:43,601 INFO:wheel:adding 'data_juicer/ops/common/special_characters.py' 2025-01-17T10:46:43,602 INFO:wheel:adding 'data_juicer/ops/deduplicator/__init__.py' 2025-01-17T10:46:43,604 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_deduplicator.py' 2025-01-17T10:46:43,606 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_minhash_deduplicator.py' 2025-01-17T10:46:43,608 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_simhash_deduplicator.py' 2025-01-17T10:46:43,609 INFO:wheel:adding 'data_juicer/ops/deduplicator/image_deduplicator.py' 2025-01-17T10:46:43,611 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_basic_deduplicator.py' 2025-01-17T10:46:43,613 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_bts_minhash_deduplicator.py' 2025-01-17T10:46:43,615 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_document_deduplicator.py' 2025-01-17T10:46:43,617 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_image_deduplicator.py' 2025-01-17T10:46:43,618 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_video_deduplicator.py' 2025-01-17T10:46:43,620 INFO:wheel:adding 'data_juicer/ops/deduplicator/video_deduplicator.py' 2025-01-17T10:46:43,623 INFO:wheel:adding 'data_juicer/ops/filter/__init__.py' 2025-01-17T10:46:43,624 INFO:wheel:adding 'data_juicer/ops/filter/alphanumeric_filter.py' 2025-01-17T10:46:43,626 INFO:wheel:adding 'data_juicer/ops/filter/audio_duration_filter.py' 2025-01-17T10:46:43,627 INFO:wheel:adding 'data_juicer/ops/filter/audio_nmf_snr_filter.py' 2025-01-17T10:46:43,629 INFO:wheel:adding 'data_juicer/ops/filter/audio_size_filter.py' 2025-01-17T10:46:43,630 INFO:wheel:adding 'data_juicer/ops/filter/average_line_length_filter.py' 2025-01-17T10:46:43,631 INFO:wheel:adding 'data_juicer/ops/filter/character_repetition_filter.py' 2025-01-17T10:46:43,633 INFO:wheel:adding 'data_juicer/ops/filter/flagged_words_filter.py' 2025-01-17T10:46:43,634 INFO:wheel:adding 'data_juicer/ops/filter/image_aesthetics_filter.py' 2025-01-17T10:46:43,635 INFO:wheel:adding 'data_juicer/ops/filter/image_aspect_ratio_filter.py' 2025-01-17T10:46:43,637 INFO:wheel:adding 'data_juicer/ops/filter/image_face_count_filter.py' 2025-01-17T10:46:43,639 INFO:wheel:adding 'data_juicer/ops/filter/image_face_ratio_filter.py' 2025-01-17T10:46:43,640 INFO:wheel:adding 'data_juicer/ops/filter/image_nsfw_filter.py' 2025-01-17T10:46:43,642 INFO:wheel:adding 'data_juicer/ops/filter/image_pair_similarity_filter.py' 2025-01-17T10:46:43,644 INFO:wheel:adding 'data_juicer/ops/filter/image_shape_filter.py' 2025-01-17T10:46:43,645 INFO:wheel:adding 'data_juicer/ops/filter/image_size_filter.py' 2025-01-17T10:46:43,646 INFO:wheel:adding 'data_juicer/ops/filter/image_text_matching_filter.py' 2025-01-17T10:46:43,648 INFO:wheel:adding 'data_juicer/ops/filter/image_text_similarity_filter.py' 2025-01-17T10:46:43,649 INFO:wheel:adding 'data_juicer/ops/filter/image_watermark_filter.py' 2025-01-17T10:46:43,651 INFO:wheel:adding 'data_juicer/ops/filter/language_id_score_filter.py' 2025-01-17T10:46:43,652 INFO:wheel:adding 'data_juicer/ops/filter/maximum_line_length_filter.py' 2025-01-17T10:46:43,653 INFO:wheel:adding 'data_juicer/ops/filter/perplexity_filter.py' 2025-01-17T10:46:43,655 INFO:wheel:adding 'data_juicer/ops/filter/phrase_grounding_recall_filter.py' 2025-01-17T10:46:43,657 INFO:wheel:adding 'data_juicer/ops/filter/special_characters_filter.py' 2025-01-17T10:46:43,658 INFO:wheel:adding 'data_juicer/ops/filter/specified_field_filter.py' 2025-01-17T10:46:43,659 INFO:wheel:adding 'data_juicer/ops/filter/specified_numeric_field_filter.py' 2025-01-17T10:46:43,661 INFO:wheel:adding 'data_juicer/ops/filter/stopwords_filter.py' 2025-01-17T10:46:43,662 INFO:wheel:adding 'data_juicer/ops/filter/suffix_filter.py' 2025-01-17T10:46:43,663 INFO:wheel:adding 'data_juicer/ops/filter/text_action_filter.py' 2025-01-17T10:46:43,665 INFO:wheel:adding 'data_juicer/ops/filter/text_entity_dependency_filter.py' 2025-01-17T10:46:43,666 INFO:wheel:adding 'data_juicer/ops/filter/text_length_filter.py' 2025-01-17T10:46:43,668 INFO:wheel:adding 'data_juicer/ops/filter/text_pair_similarity_filter.py' 2025-01-17T10:46:43,669 INFO:wheel:adding 'data_juicer/ops/filter/token_num_filter.py' 2025-01-17T10:46:43,671 INFO:wheel:adding 'data_juicer/ops/filter/video_aesthetics_filter.py' 2025-01-17T10:46:43,673 INFO:wheel:adding 'data_juicer/ops/filter/video_aspect_ratio_filter.py' 2025-01-17T10:46:43,674 INFO:wheel:adding 'data_juicer/ops/filter/video_duration_filter.py' 2025-01-17T10:46:43,676 INFO:wheel:adding 'data_juicer/ops/filter/video_frames_text_similarity_filter.py' 2025-01-17T10:46:43,678 INFO:wheel:adding 'data_juicer/ops/filter/video_motion_score_filter.py' 2025-01-17T10:46:43,679 INFO:wheel:adding 'data_juicer/ops/filter/video_motion_score_raft_filter.py' 2025-01-17T10:46:43,680 INFO:wheel:adding 'data_juicer/ops/filter/video_nsfw_filter.py' 2025-01-17T10:46:43,682 INFO:wheel:adding 'data_juicer/ops/filter/video_ocr_area_ratio_filter.py' 2025-01-17T10:46:43,683 INFO:wheel:adding 'data_juicer/ops/filter/video_resolution_filter.py' 2025-01-17T10:46:43,685 INFO:wheel:adding 'data_juicer/ops/filter/video_tagging_from_frames_filter.py' 2025-01-17T10:46:43,687 INFO:wheel:adding 'data_juicer/ops/filter/video_watermark_filter.py' 2025-01-17T10:46:43,688 INFO:wheel:adding 'data_juicer/ops/filter/word_repetition_filter.py' 2025-01-17T10:46:43,689 INFO:wheel:adding 'data_juicer/ops/filter/words_num_filter.py' 2025-01-17T10:46:43,691 INFO:wheel:adding 'data_juicer/ops/grouper/__init__.py' 2025-01-17T10:46:43,692 INFO:wheel:adding 'data_juicer/ops/grouper/key_value_grouper.py' 2025-01-17T10:46:43,694 INFO:wheel:adding 'data_juicer/ops/grouper/naive_grouper.py' 2025-01-17T10:46:43,695 INFO:wheel:adding 'data_juicer/ops/grouper/naive_reverse_grouper.py' 2025-01-17T10:46:43,699 INFO:wheel:adding 'data_juicer/ops/mapper/__init__.py' 2025-01-17T10:46:43,700 INFO:wheel:adding 'data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py' 2025-01-17T10:46:43,702 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_qa_mapper.py' 2025-01-17T10:46:43,703 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_query_mapper.py' 2025-01-17T10:46:43,705 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_response_mapper.py' 2025-01-17T10:46:43,706 INFO:wheel:adding 'data_juicer/ops/mapper/chinese_convert_mapper.py' 2025-01-17T10:46:43,707 INFO:wheel:adding 'data_juicer/ops/mapper/clean_copyright_mapper.py' 2025-01-17T10:46:43,709 INFO:wheel:adding 'data_juicer/ops/mapper/clean_email_mapper.py' 2025-01-17T10:46:43,711 INFO:wheel:adding 'data_juicer/ops/mapper/clean_html_mapper.py' 2025-01-17T10:46:43,712 INFO:wheel:adding 'data_juicer/ops/mapper/clean_ip_mapper.py' 2025-01-17T10:46:43,713 INFO:wheel:adding 'data_juicer/ops/mapper/clean_links_mapper.py' 2025-01-17T10:46:43,715 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_intent_detection_mapper.py' 2025-01-17T10:46:43,717 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_sentiment_detection_mapper.py' 2025-01-17T10:46:43,718 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_sentiment_intensity_mapper.py' 2025-01-17T10:46:43,720 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_topic_detection_mapper.py' 2025-01-17T10:46:43,722 INFO:wheel:adding 'data_juicer/ops/mapper/expand_macro_mapper.py' 2025-01-17T10:46:43,723 INFO:wheel:adding 'data_juicer/ops/mapper/extract_entity_attribute_mapper.py' 2025-01-17T10:46:43,727 INFO:wheel:adding 'data_juicer/ops/mapper/extract_entity_relation_mapper.py' 2025-01-17T10:46:43,728 INFO:wheel:adding 'data_juicer/ops/mapper/extract_event_mapper.py' 2025-01-17T10:46:43,730 INFO:wheel:adding 'data_juicer/ops/mapper/extract_keyword_mapper.py' 2025-01-17T10:46:43,732 INFO:wheel:adding 'data_juicer/ops/mapper/extract_nickname_mapper.py' 2025-01-17T10:46:43,733 INFO:wheel:adding 'data_juicer/ops/mapper/extract_support_text_mapper.py' 2025-01-17T10:46:43,735 INFO:wheel:adding 'data_juicer/ops/mapper/fix_unicode_mapper.py' 2025-01-17T10:46:43,737 INFO:wheel:adding 'data_juicer/ops/mapper/generate_qa_from_examples_mapper.py' 2025-01-17T10:46:43,738 INFO:wheel:adding 'data_juicer/ops/mapper/generate_qa_from_text_mapper.py' 2025-01-17T10:46:43,740 INFO:wheel:adding 'data_juicer/ops/mapper/image_blur_mapper.py' 2025-01-17T10:46:43,742 INFO:wheel:adding 'data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py' 2025-01-17T10:46:43,744 INFO:wheel:adding 'data_juicer/ops/mapper/image_captioning_mapper.py' 2025-01-17T10:46:43,746 INFO:wheel:adding 'data_juicer/ops/mapper/image_diffusion_mapper.py' 2025-01-17T10:46:43,747 INFO:wheel:adding 'data_juicer/ops/mapper/image_face_blur_mapper.py' 2025-01-17T10:46:43,749 INFO:wheel:adding 'data_juicer/ops/mapper/image_segment_mapper.py' 2025-01-17T10:46:43,751 INFO:wheel:adding 'data_juicer/ops/mapper/image_tagging_mapper.py' 2025-01-17T10:46:43,752 INFO:wheel:adding 'data_juicer/ops/mapper/mllm_mapper.py' 2025-01-17T10:46:43,754 INFO:wheel:adding 'data_juicer/ops/mapper/nlpaug_en_mapper.py' 2025-01-17T10:46:43,755 INFO:wheel:adding 'data_juicer/ops/mapper/nlpcda_zh_mapper.py' 2025-01-17T10:46:43,757 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_qa_mapper.py' 2025-01-17T10:46:43,758 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_query_mapper.py' 2025-01-17T10:46:43,759 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_response_mapper.py' 2025-01-17T10:46:43,761 INFO:wheel:adding 'data_juicer/ops/mapper/pair_preference_mapper.py' 2025-01-17T10:46:43,762 INFO:wheel:adding 'data_juicer/ops/mapper/punctuation_normalization_mapper.py' 2025-01-17T10:46:43,764 INFO:wheel:adding 'data_juicer/ops/mapper/python_file_mapper.py' 2025-01-17T10:46:43,765 INFO:wheel:adding 'data_juicer/ops/mapper/python_lambda_mapper.py' 2025-01-17T10:46:43,766 INFO:wheel:adding 'data_juicer/ops/mapper/query_intent_detection_mapper.py' 2025-01-17T10:46:43,768 INFO:wheel:adding 'data_juicer/ops/mapper/query_sentiment_detection_mapper.py' 2025-01-17T10:46:43,769 INFO:wheel:adding 'data_juicer/ops/mapper/query_topic_detection_mapper.py' 2025-01-17T10:46:43,771 INFO:wheel:adding 'data_juicer/ops/mapper/relation_identity_mapper.py' 2025-01-17T10:46:43,772 INFO:wheel:adding 'data_juicer/ops/mapper/remove_bibliography_mapper.py' 2025-01-17T10:46:43,773 INFO:wheel:adding 'data_juicer/ops/mapper/remove_comments_mapper.py' 2025-01-17T10:46:43,775 INFO:wheel:adding 'data_juicer/ops/mapper/remove_header_mapper.py' 2025-01-17T10:46:43,776 INFO:wheel:adding 'data_juicer/ops/mapper/remove_long_words_mapper.py' 2025-01-17T10:46:43,777 INFO:wheel:adding 'data_juicer/ops/mapper/remove_non_chinese_character_mapper.py' 2025-01-17T10:46:43,779 INFO:wheel:adding 'data_juicer/ops/mapper/remove_repeat_sentences_mapper.py' 2025-01-17T10:46:43,780 INFO:wheel:adding 'data_juicer/ops/mapper/remove_specific_chars_mapper.py' 2025-01-17T10:46:43,781 INFO:wheel:adding 'data_juicer/ops/mapper/remove_table_text_mapper.py' 2025-01-17T10:46:43,783 INFO:wheel:adding 'data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py' 2025-01-17T10:46:43,784 INFO:wheel:adding 'data_juicer/ops/mapper/replace_content_mapper.py' 2025-01-17T10:46:43,790 INFO:wheel:adding 'data_juicer/ops/mapper/sdxl_prompt2prompt_mapper.py' 2025-01-17T10:46:43,792 INFO:wheel:adding 'data_juicer/ops/mapper/sentence_augmentation_mapper.py' 2025-01-17T10:46:43,793 INFO:wheel:adding 'data_juicer/ops/mapper/sentence_split_mapper.py' 2025-01-17T10:46:43,795 INFO:wheel:adding 'data_juicer/ops/mapper/text_chunk_mapper.py' 2025-01-17T10:46:43,796 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_audio_mapper.py' 2025-01-17T10:46:43,798 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_frames_mapper.py' 2025-01-17T10:46:43,800 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py' 2025-01-17T10:46:43,803 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_video_mapper.py' 2025-01-17T10:46:43,804 INFO:wheel:adding 'data_juicer/ops/mapper/video_extract_frames_mapper.py' 2025-01-17T10:46:43,806 INFO:wheel:adding 'data_juicer/ops/mapper/video_face_blur_mapper.py' 2025-01-17T10:46:43,807 INFO:wheel:adding 'data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py' 2025-01-17T10:46:43,809 INFO:wheel:adding 'data_juicer/ops/mapper/video_remove_watermark_mapper.py' 2025-01-17T10:46:43,810 INFO:wheel:adding 'data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py' 2025-01-17T10:46:43,812 INFO:wheel:adding 'data_juicer/ops/mapper/video_resize_resolution_mapper.py' 2025-01-17T10:46:43,814 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_duration_mapper.py' 2025-01-17T10:46:43,815 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_key_frame_mapper.py' 2025-01-17T10:46:43,817 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_scene_mapper.py' 2025-01-17T10:46:43,818 INFO:wheel:adding 'data_juicer/ops/mapper/video_tagging_from_audio_mapper.py' 2025-01-17T10:46:43,820 INFO:wheel:adding 'data_juicer/ops/mapper/video_tagging_from_frames_mapper.py' 2025-01-17T10:46:43,821 INFO:wheel:adding 'data_juicer/ops/mapper/whitespace_normalization_mapper.py' 2025-01-17T10:46:43,823 INFO:wheel:adding 'data_juicer/ops/selector/__init__.py' 2025-01-17T10:46:43,824 INFO:wheel:adding 'data_juicer/ops/selector/frequency_specified_field_selector.py' 2025-01-17T10:46:43,826 INFO:wheel:adding 'data_juicer/ops/selector/random_selector.py' 2025-01-17T10:46:43,827 INFO:wheel:adding 'data_juicer/ops/selector/range_specified_field_selector.py' 2025-01-17T10:46:43,829 INFO:wheel:adding 'data_juicer/ops/selector/tags_specified_field_selector.py' 2025-01-17T10:46:43,831 INFO:wheel:adding 'data_juicer/ops/selector/topk_specified_field_selector.py' 2025-01-17T10:46:43,832 INFO:wheel:adding 'data_juicer/tools/__init__.py' 2025-01-17T10:46:43,833 INFO:wheel:adding 'data_juicer/tools/analyze_data.py' 2025-01-17T10:46:43,835 INFO:wheel:adding 'data_juicer/tools/dj_install.py' 2025-01-17T10:46:43,836 INFO:wheel:adding 'data_juicer/tools/process_data.py' 2025-01-17T10:46:43,837 INFO:wheel:adding 'data_juicer/tools/sandbox_starter.py' 2025-01-17T10:46:43,839 INFO:wheel:adding 'data_juicer/utils/__init__.py' 2025-01-17T10:46:43,841 INFO:wheel:adding 'data_juicer/utils/asset_utils.py' 2025-01-17T10:46:43,842 INFO:wheel:adding 'data_juicer/utils/auto_install_mapping.py' 2025-01-17T10:46:43,843 INFO:wheel:adding 'data_juicer/utils/auto_install_utils.py' 2025-01-17T10:46:43,845 INFO:wheel:adding 'data_juicer/utils/availability_utils.py' 2025-01-17T10:46:43,846 INFO:wheel:adding 'data_juicer/utils/cache_utils.py' 2025-01-17T10:46:43,847 INFO:wheel:adding 'data_juicer/utils/ckpt_utils.py' 2025-01-17T10:46:43,849 INFO:wheel:adding 'data_juicer/utils/common_utils.py' 2025-01-17T10:46:43,851 INFO:wheel:adding 'data_juicer/utils/compress.py' 2025-01-17T10:46:43,853 INFO:wheel:adding 'data_juicer/utils/constant.py' 2025-01-17T10:46:43,855 INFO:wheel:adding 'data_juicer/utils/file_utils.py' 2025-01-17T10:46:43,856 INFO:wheel:adding 'data_juicer/utils/fingerprint_utils.py' 2025-01-17T10:46:43,857 INFO:wheel:adding 'data_juicer/utils/lazy_loader.py' 2025-01-17T10:46:43,859 INFO:wheel:adding 'data_juicer/utils/logger_utils.py' 2025-01-17T10:46:43,863 INFO:wheel:adding 'data_juicer/utils/mm_utils.py' 2025-01-17T10:46:43,867 INFO:wheel:adding 'data_juicer/utils/model_utils.py' 2025-01-17T10:46:43,869 INFO:wheel:adding 'data_juicer/utils/process_utils.py' 2025-01-17T10:46:43,870 INFO:wheel:adding 'data_juicer/utils/registry.py' 2025-01-17T10:46:43,871 INFO:wheel:adding 'data_juicer/utils/resource_utils.py' 2025-01-17T10:46:43,873 INFO:wheel:adding 'data_juicer/utils/unittest_utils.py' 2025-01-17T10:46:43,876 INFO:wheel:adding 'py_data_juicer-1.1.0.dist-info/LICENSE' 2025-01-17T10:46:43,881 INFO:wheel:adding 'py_data_juicer-1.1.0.dist-info/METADATA' 2025-01-17T10:46:43,882 INFO:wheel:adding 'py_data_juicer-1.1.0.dist-info/WHEEL' 2025-01-17T10:46:43,883 INFO:wheel:adding 'py_data_juicer-1.1.0.dist-info/entry_points.txt' 2025-01-17T10:46:43,883 INFO:wheel:adding 'py_data_juicer-1.1.0.dist-info/top_level.txt' 2025-01-17T10:46:43,887 INFO:wheel:adding 'py_data_juicer-1.1.0.dist-info/RECORD' 2025-01-17T10:46:43,895 INFO:root:removing build/bdist.linux-armv7l/wheel 2025-01-17T10:46:44,036 Building wheel for py-data-juicer (setup.py): finished with status 'done' 2025-01-17T10:46:44,043 Created wheel for py-data-juicer: filename=py_data_juicer-1.1.0-py3-none-any.whl size=412215 sha256=d2994ed187dd328adc3b4751b862a32f4847197a82b9ddd5dd4ed97960aac8d1 2025-01-17T10:46:44,044 Stored in directory: /tmp/pip-ephem-wheel-cache-89sxn7kp/wheels/e8/19/c5/1d8206d31691898bcda1759849ea5123012c43eb6a4420e5f5 2025-01-17T10:46:44,066 Successfully built py-data-juicer 2025-01-17T10:46:44,079 Removed build tracker: '/tmp/pip-build-tracker-9kl3el4x'