2025-03-28T21:37:33,475 Created temporary directory: /tmp/pip-build-tracker-q2225lfa 2025-03-28T21:37:33,476 Initialized build tracking at /tmp/pip-build-tracker-q2225lfa 2025-03-28T21:37:33,477 Created build tracker: /tmp/pip-build-tracker-q2225lfa 2025-03-28T21:37:33,477 Entered build tracker: /tmp/pip-build-tracker-q2225lfa 2025-03-28T21:37:33,478 Created temporary directory: /tmp/pip-wheel-u9z4vtxv 2025-03-28T21:37:33,482 Created temporary directory: /tmp/pip-ephem-wheel-cache-7flixiid 2025-03-28T21:37:33,527 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2025-03-28T21:37:33,530 2 location(s) to search for versions of py-data-juicer: 2025-03-28T21:37:33,530 * https://pypi.org/simple/py-data-juicer/ 2025-03-28T21:37:33,530 * https://www.piwheels.org/simple/py-data-juicer/ 2025-03-28T21:37:33,531 Fetching project page and analyzing links: https://pypi.org/simple/py-data-juicer/ 2025-03-28T21:37:33,531 Getting page https://pypi.org/simple/py-data-juicer/ 2025-03-28T21:37:33,533 Found index url https://pypi.org/simple/ 2025-03-28T21:37:33,753 Fetched page https://pypi.org/simple/py-data-juicer/ as application/vnd.pypi.simple.v1+json 2025-03-28T21:37:33,759 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/e3/d3/d0724d922e7c55a0664485fdf642124bdcd801df2697e29c9463c4360958/py_data_juicer-0.1.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,759 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/fd/c6/e1428310bf534319ddc2e362c6eee21985f097e70cedc0e9cd3e914de826/py_data_juicer-0.1.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,760 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/67/70/731e349d2a92bf59a767230b956665dd27ea24a7a948d4a1710154d77a24/py_data_juicer-0.1.2-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,760 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/66/08/f584efbdf8277a061ce73c9befc33fec813cbd08020761af2380dd331692/py_data_juicer-0.1.3-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,762 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/69/20/9862dfe7a94f10caa0e6387834d1265420613345c7adadd41a3ced7230e8/py_data_juicer-0.2.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,762 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/05/34/33fb401d350f1a66cdaefc224419f9d01484722829f5b0c71ef59a8365a4/py_data_juicer-1.0.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,763 Found link https://files.pythonhosted.org/packages/49/e9/cfb994255490c36554048e0b3956858e61f5a75ffacfde20849f31f04ef8/py_data_juicer-1.0.0.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.0 2025-03-28T21:37:33,764 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/27/37/150e2198f14349fdd6b647a63feaede809e21dbfc4617aba487f18049828/py_data_juicer-1.0.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,764 Found link https://files.pythonhosted.org/packages/4a/60/dadcbe4337a76d8f98022b60d142ac31073d568c9e06a5461b3014d16fc6/py_data_juicer-1.0.1.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.1 2025-03-28T21:37:33,765 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/10/dd/f9cadd6ed2f19c4f94e61a3fd4bb5bb2029c20b2cf61ec86c3a05f89f74b/py_data_juicer-1.0.2-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,766 Found link https://files.pythonhosted.org/packages/95/8f/c0447ce091ca0b86283e8074a3cb350572a5436ca13dceb288887a0de332/py_data_juicer-1.0.2.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.2 2025-03-28T21:37:33,766 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/1f/dd/0f1804c5cfce50c52c8b60a1b710f30d893f681e42a6acc41bfd29a20059/py_data_juicer-1.0.3-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,767 Found link https://files.pythonhosted.org/packages/6e/d0/eefb1ca00cd4c8e62c0e8f62a1b5357b961ce50c5d2647107f51f68bf128/py_data_juicer-1.0.3.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.0.3 2025-03-28T21:37:33,768 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/5b/06/5df5581f724d49731b91db0b7c65fdde739358617af9e7ba9916c43f2a76/py_data_juicer-1.1.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,769 Found link https://files.pythonhosted.org/packages/69/43/f20a50fdbfc0ba44a4816d3c7f9bfcfaf55682ffebf78a10eeff6b40feeb/py_data_juicer-1.1.0.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.1.0 2025-03-28T21:37:33,769 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/a0/90/a95bb5b60200125b9cbb29bf1c39b706c3e5f50b247d3599386f7be973ea/py_data_juicer-1.1.0.post1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,770 Found link https://files.pythonhosted.org/packages/79/f8/3e29fb4eeeda4413bc746b0a661bc61d5786f8bf67ae9071976550fd0b41/py_data_juicer-1.1.0.post1.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.1.0.post1 2025-03-28T21:37:33,770 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/9d/ca/e32ecfafde96a367dcb63a4c4ee104bc5e952205c55169fd1120df0e86ee/py_data_juicer-1.2.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,771 Found link https://files.pythonhosted.org/packages/3d/bc/607884c148a0b6bf2198960166743e95818d230e970cbcdbd69d02758541/py_data_juicer-1.2.0.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.2.0 2025-03-28T21:37:33,771 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/80/a2/4788daf084637e94c05615f237298e1d7fe13753f7622907ce0904a4b18c/py_data_juicer-1.2.1-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,772 Found link https://files.pythonhosted.org/packages/df/b6/0ca40521ecc5d3a79f8e3614af2974f4a50ae0c175fee658c8f2b091eb6b/py_data_juicer-1.2.1.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.2.1 2025-03-28T21:37:33,773 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/2d/ec/75b61b05a4b19fe2e425f10b0e7f023c13d404be4ec7aa7b39ad85f18e3f/py_data_juicer-1.2.2-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,773 Found link https://files.pythonhosted.org/packages/8e/01/e4ad384e1c25acb029fd433f6bcc5d0c146cf729bd3801308cc681354ac1/py_data_juicer-1.2.2.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.2.2 2025-03-28T21:37:33,774 Skipping link: No binaries permitted for py-data-juicer: https://files.pythonhosted.org/packages/a6/3a/6b6ec164c0b6270a2dde6c03c449387f21e3f72d98ecc2e1bb98d7c6224f/py_data_juicer-1.3.0-py3-none-any.whl (from https://pypi.org/simple/py-data-juicer/) 2025-03-28T21:37:33,775 Found link https://files.pythonhosted.org/packages/6c/56/b2fccdf35cfbc8bd34f90baa136bd86a81205f35e4ed1057503fb33a1ce8/py_data_juicer-1.3.0.tar.gz (from https://pypi.org/simple/py-data-juicer/), version: 1.3.0 2025-03-28T21:37:33,775 Fetching project page and analyzing links: https://www.piwheels.org/simple/py-data-juicer/ 2025-03-28T21:37:33,776 Getting page https://www.piwheels.org/simple/py-data-juicer/ 2025-03-28T21:37:33,777 Found index url https://www.piwheels.org/simple/ 2025-03-28T21:37:33,957 Fetched page https://www.piwheels.org/simple/py-data-juicer/ as text/html 2025-03-28T21:37:33,961 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.2.2-py3-none-any.whl#sha256=7cb9219b41ba63aaeee2dda88f3303d1e05ed75733902a76d62bedb845406f87 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-03-28T21:37:33,962 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.2.1-py3-none-any.whl#sha256=ddf1558f432f3ba201ed207faa7a45ac9c9008c15240f4495c315b11c72b8872 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-03-28T21:37:33,963 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.2.0-py3-none-any.whl#sha256=91f27adcba48929c9c2f59ff369a2712a2386dc8795e51481069dcd4793b2754 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-03-28T21:37:33,963 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.1.0.post1-py3-none-any.whl#sha256=dc638ddb7c2a90b1970855a7393432a06b8bc214943edf76ed0aa55fb3ba8b76 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-03-28T21:37:33,964 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.1.0-py3-none-any.whl#sha256=d2994ed187dd328adc3b4751b862a32f4847197a82b9ddd5dd4ed97960aac8d1 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-03-28T21:37:33,965 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.3-py3-none-any.whl#sha256=ec09d1fb68352b5fb204a2cab5cac661f02323ef0cbc6a52833ea83418e55a3a (from https://www.piwheels.org/simple/py-data-juicer/) 2025-03-28T21:37:33,966 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.2-py3-none-any.whl#sha256=69bc91a28885bd8df218510cd9c0198397f76eaab0c7fbf9726181da70734781 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-03-28T21:37:33,967 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.1-py3-none-any.whl#sha256=2396c39923ab13a9fe17fe6523734f2ff41b3edb2c97fcc70539a5761e0465f8 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-03-28T21:37:33,967 Skipping link: No binaries permitted for py-data-juicer: https://www.piwheels.org/simple/py-data-juicer/py_data_juicer-1.0.0-py3-none-any.whl#sha256=7cd007456933447474ec9faf5357b741a63e7a0c0dc6230e40bf4498eb1ef554 (from https://www.piwheels.org/simple/py-data-juicer/) 2025-03-28T21:37:33,968 Skipping link: not a file: https://www.piwheels.org/simple/py-data-juicer/ 2025-03-28T21:37:33,969 Skipping link: not a file: https://pypi.org/simple/py-data-juicer/ 2025-03-28T21:37:33,991 Given no hashes to check 1 links for project 'py-data-juicer': discarding no candidates 2025-03-28T21:37:33,997 Collecting py-data-juicer==1.3.0 2025-03-28T21:37:34,004 Created temporary directory: /tmp/pip-unpack-18mrov6w 2025-03-28T21:37:34,239 Downloading py_data_juicer-1.3.0.tar.gz (323 kB) 2025-03-28T21:37:34,787 Added py-data-juicer==1.3.0 from https://files.pythonhosted.org/packages/6c/56/b2fccdf35cfbc8bd34f90baa136bd86a81205f35e4ed1057503fb33a1ce8/py_data_juicer-1.3.0.tar.gz to build tracker '/tmp/pip-build-tracker-q2225lfa' 2025-03-28T21:37:34,790 Running setup.py (path:/tmp/pip-wheel-u9z4vtxv/py-data-juicer_33514d964d3b43198a1f71a340684fe2/setup.py) egg_info for package py-data-juicer 2025-03-28T21:37:34,791 Created temporary directory: /tmp/pip-pip-egg-info-7gz8ahsk 2025-03-28T21:37:34,791 Preparing metadata (setup.py): started 2025-03-28T21:37:34,793 Running command python setup.py egg_info 2025-03-28T21:37:35,266 WARNING:root:target file does not exist: environments/minimal_requires.txt 2025-03-28T21:37:35,266 WARNING:root:target file does not exist: environments/science_requires.txt 2025-03-28T21:37:35,267 WARNING:root:target file does not exist: environments/dist_requires.txt 2025-03-28T21:37:35,267 WARNING:root:target file does not exist: environments/dev_requires.txt 2025-03-28T21:37:35,268 WARNING:root:target file does not exist: environments/preprocess_requires.txt 2025-03-28T21:37:35,268 WARNING:root:target file does not exist: environments/quality_classifier_requires.txt 2025-03-28T21:37:35,269 WARNING:root:target file does not exist: environments/sandbox_requires.txt 2025-03-28T21:37:35,888 INFO:root:running egg_info 2025-03-28T21:37:35,918 INFO:root:creating /tmp/pip-pip-egg-info-7gz8ahsk/py_data_juicer.egg-info 2025-03-28T21:37:35,919 INFO:root:writing /tmp/pip-pip-egg-info-7gz8ahsk/py_data_juicer.egg-info/PKG-INFO 2025-03-28T21:37:35,923 INFO:root:writing dependency_links to /tmp/pip-pip-egg-info-7gz8ahsk/py_data_juicer.egg-info/dependency_links.txt 2025-03-28T21:37:35,924 INFO:root:writing entry points to /tmp/pip-pip-egg-info-7gz8ahsk/py_data_juicer.egg-info/entry_points.txt 2025-03-28T21:37:35,926 INFO:root:writing requirements to /tmp/pip-pip-egg-info-7gz8ahsk/py_data_juicer.egg-info/requires.txt 2025-03-28T21:37:35,927 INFO:root:writing top-level names to /tmp/pip-pip-egg-info-7gz8ahsk/py_data_juicer.egg-info/top_level.txt 2025-03-28T21:37:35,929 INFO:root:writing manifest file '/tmp/pip-pip-egg-info-7gz8ahsk/py_data_juicer.egg-info/SOURCES.txt' 2025-03-28T21:37:36,044 INFO:root:reading manifest file '/tmp/pip-pip-egg-info-7gz8ahsk/py_data_juicer.egg-info/SOURCES.txt' 2025-03-28T21:37:36,046 INFO:root:adding license file 'LICENSE' 2025-03-28T21:37:36,055 INFO:root:writing manifest file '/tmp/pip-pip-egg-info-7gz8ahsk/py_data_juicer.egg-info/SOURCES.txt' 2025-03-28T21:37:36,149 Preparing metadata (setup.py): finished with status 'done' 2025-03-28T21:37:36,155 Source in /tmp/pip-wheel-u9z4vtxv/py-data-juicer_33514d964d3b43198a1f71a340684fe2 has version 1.3.0, which satisfies requirement py-data-juicer==1.3.0 from https://files.pythonhosted.org/packages/6c/56/b2fccdf35cfbc8bd34f90baa136bd86a81205f35e4ed1057503fb33a1ce8/py_data_juicer-1.3.0.tar.gz 2025-03-28T21:37:36,156 Removed py-data-juicer==1.3.0 from https://files.pythonhosted.org/packages/6c/56/b2fccdf35cfbc8bd34f90baa136bd86a81205f35e4ed1057503fb33a1ce8/py_data_juicer-1.3.0.tar.gz from build tracker '/tmp/pip-build-tracker-q2225lfa' 2025-03-28T21:37:36,164 Created temporary directory: /tmp/pip-unpack-z2elashu 2025-03-28T21:37:36,165 Created temporary directory: /tmp/pip-unpack-n25b5uo7 2025-03-28T21:37:36,166 Building wheels for collected packages: py-data-juicer 2025-03-28T21:37:36,170 Created temporary directory: /tmp/pip-wheel-6xj1jnpg 2025-03-28T21:37:36,171 Building wheel for py-data-juicer (setup.py): started 2025-03-28T21:37:36,172 Destination directory: /tmp/pip-wheel-6xj1jnpg 2025-03-28T21:37:36,173 Running command python setup.py bdist_wheel 2025-03-28T21:37:36,636 WARNING:root:target file does not exist: environments/minimal_requires.txt 2025-03-28T21:37:36,636 WARNING:root:target file does not exist: environments/science_requires.txt 2025-03-28T21:37:36,636 WARNING:root:target file does not exist: environments/dist_requires.txt 2025-03-28T21:37:36,637 WARNING:root:target file does not exist: environments/dev_requires.txt 2025-03-28T21:37:36,638 WARNING:root:target file does not exist: environments/preprocess_requires.txt 2025-03-28T21:37:36,638 WARNING:root:target file does not exist: environments/quality_classifier_requires.txt 2025-03-28T21:37:36,639 WARNING:root:target file does not exist: environments/sandbox_requires.txt 2025-03-28T21:37:37,215 INFO:root:running bdist_wheel 2025-03-28T21:37:37,351 INFO:root:running build 2025-03-28T21:37:37,352 INFO:root:running build_py 2025-03-28T21:37:37,384 INFO:root:creating build/lib/data_juicer 2025-03-28T21:37:37,387 INFO:root:copying data_juicer/__init__.py -> build/lib/data_juicer 2025-03-28T21:37:37,389 INFO:root:creating build/lib/data_juicer/format 2025-03-28T21:37:37,390 INFO:root:copying data_juicer/format/formatter.py -> build/lib/data_juicer/format 2025-03-28T21:37:37,393 INFO:root:copying data_juicer/format/text_formatter.py -> build/lib/data_juicer/format 2025-03-28T21:37:37,395 INFO:root:copying data_juicer/format/parquet_formatter.py -> build/lib/data_juicer/format 2025-03-28T21:37:37,397 INFO:root:copying data_juicer/format/empty_formatter.py -> build/lib/data_juicer/format 2025-03-28T21:37:37,399 INFO:root:copying data_juicer/format/load.py -> build/lib/data_juicer/format 2025-03-28T21:37:37,401 INFO:root:copying data_juicer/format/tsv_formatter.py -> build/lib/data_juicer/format 2025-03-28T21:37:37,403 INFO:root:copying data_juicer/format/csv_formatter.py -> build/lib/data_juicer/format 2025-03-28T21:37:37,405 INFO:root:copying data_juicer/format/__init__.py -> build/lib/data_juicer/format 2025-03-28T21:37:37,406 INFO:root:copying data_juicer/format/json_formatter.py -> build/lib/data_juicer/format 2025-03-28T21:37:37,409 INFO:root:creating build/lib/data_juicer/analysis 2025-03-28T21:37:37,410 INFO:root:copying data_juicer/analysis/collector.py -> build/lib/data_juicer/analysis 2025-03-28T21:37:37,412 INFO:root:copying data_juicer/analysis/measure.py -> build/lib/data_juicer/analysis 2025-03-28T21:37:37,414 INFO:root:copying data_juicer/analysis/draw.py -> build/lib/data_juicer/analysis 2025-03-28T21:37:37,416 INFO:root:copying data_juicer/analysis/column_wise_analysis.py -> build/lib/data_juicer/analysis 2025-03-28T21:37:37,419 INFO:root:copying data_juicer/analysis/diversity_analysis.py -> build/lib/data_juicer/analysis 2025-03-28T21:37:37,421 INFO:root:copying data_juicer/analysis/__init__.py -> build/lib/data_juicer/analysis 2025-03-28T21:37:37,423 INFO:root:copying data_juicer/analysis/overall_analysis.py -> build/lib/data_juicer/analysis 2025-03-28T21:37:37,425 INFO:root:creating build/lib/data_juicer/download 2025-03-28T21:37:37,426 INFO:root:copying data_juicer/download/arxiv.py -> build/lib/data_juicer/download 2025-03-28T21:37:37,429 INFO:root:copying data_juicer/download/commoncrawl.py -> build/lib/data_juicer/download 2025-03-28T21:37:37,431 INFO:root:copying data_juicer/download/downloader.py -> build/lib/data_juicer/download 2025-03-28T21:37:37,433 INFO:root:copying data_juicer/download/__init__.py -> build/lib/data_juicer/download 2025-03-28T21:37:37,434 INFO:root:copying data_juicer/download/wikipedia.py -> build/lib/data_juicer/download 2025-03-28T21:37:37,438 INFO:root:creating build/lib/data_juicer/config 2025-03-28T21:37:37,439 INFO:root:copying data_juicer/config/config.py -> build/lib/data_juicer/config 2025-03-28T21:37:37,442 INFO:root:copying data_juicer/config/__init__.py -> build/lib/data_juicer/config 2025-03-28T21:37:37,445 INFO:root:creating build/lib/data_juicer/utils 2025-03-28T21:37:37,446 INFO:root:copying data_juicer/utils/compress.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,449 INFO:root:copying data_juicer/utils/logger_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,451 INFO:root:copying data_juicer/utils/model_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,454 INFO:root:copying data_juicer/utils/resource_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,456 INFO:root:copying data_juicer/utils/auto_install_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,458 INFO:root:copying data_juicer/utils/process_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,460 INFO:root:copying data_juicer/utils/availability_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,462 INFO:root:copying data_juicer/utils/unittest_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,464 INFO:root:copying data_juicer/utils/asset_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,466 INFO:root:copying data_juicer/utils/cache_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,468 INFO:root:copying data_juicer/utils/mm_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,471 INFO:root:copying data_juicer/utils/constant.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,473 INFO:root:copying data_juicer/utils/auto_install_mapping.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,476 INFO:root:copying data_juicer/utils/file_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,478 INFO:root:copying data_juicer/utils/__init__.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,480 INFO:root:copying data_juicer/utils/lazy_loader.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,482 INFO:root:copying data_juicer/utils/sample.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,483 INFO:root:copying data_juicer/utils/common_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,485 INFO:root:copying data_juicer/utils/ckpt_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,488 INFO:root:copying data_juicer/utils/registry.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,490 INFO:root:copying data_juicer/utils/fingerprint_utils.py -> build/lib/data_juicer/utils 2025-03-28T21:37:37,493 INFO:root:creating build/lib/data_juicer/ops 2025-03-28T21:37:37,494 INFO:root:copying data_juicer/ops/base_op.py -> build/lib/data_juicer/ops 2025-03-28T21:37:37,496 INFO:root:copying data_juicer/ops/load.py -> build/lib/data_juicer/ops 2025-03-28T21:37:37,498 INFO:root:copying data_juicer/ops/__init__.py -> build/lib/data_juicer/ops 2025-03-28T21:37:37,499 INFO:root:copying data_juicer/ops/op_fusion.py -> build/lib/data_juicer/ops 2025-03-28T21:37:37,502 INFO:root:creating build/lib/data_juicer/core 2025-03-28T21:37:37,503 INFO:root:copying data_juicer/core/tracer.py -> build/lib/data_juicer/core 2025-03-28T21:37:37,505 INFO:root:copying data_juicer/core/exporter.py -> build/lib/data_juicer/core 2025-03-28T21:37:37,507 INFO:root:copying data_juicer/core/analyzer.py -> build/lib/data_juicer/core 2025-03-28T21:37:37,509 INFO:root:copying data_juicer/core/monitor.py -> build/lib/data_juicer/core 2025-03-28T21:37:37,512 INFO:root:copying data_juicer/core/__init__.py -> build/lib/data_juicer/core 2025-03-28T21:37:37,513 INFO:root:copying data_juicer/core/adapter.py -> build/lib/data_juicer/core 2025-03-28T21:37:37,516 INFO:root:creating build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,517 INFO:root:copying data_juicer/ops/deduplicator/video_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,520 INFO:root:copying data_juicer/ops/deduplicator/ray_basic_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,522 INFO:root:copying data_juicer/ops/deduplicator/ray_image_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,523 INFO:root:copying data_juicer/ops/deduplicator/document_minhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,526 INFO:root:copying data_juicer/ops/deduplicator/document_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,528 INFO:root:copying data_juicer/ops/deduplicator/__init__.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,530 INFO:root:copying data_juicer/ops/deduplicator/ray_bts_minhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,532 INFO:root:copying data_juicer/ops/deduplicator/document_simhash_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,534 INFO:root:copying data_juicer/ops/deduplicator/ray_video_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,536 INFO:root:copying data_juicer/ops/deduplicator/image_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,538 INFO:root:copying data_juicer/ops/deduplicator/ray_document_deduplicator.py -> build/lib/data_juicer/ops/deduplicator 2025-03-28T21:37:37,541 INFO:root:creating build/lib/data_juicer/ops/grouper 2025-03-28T21:37:37,542 INFO:root:copying data_juicer/ops/grouper/key_value_grouper.py -> build/lib/data_juicer/ops/grouper 2025-03-28T21:37:37,544 INFO:root:copying data_juicer/ops/grouper/naive_grouper.py -> build/lib/data_juicer/ops/grouper 2025-03-28T21:37:37,546 INFO:root:copying data_juicer/ops/grouper/naive_reverse_grouper.py -> build/lib/data_juicer/ops/grouper 2025-03-28T21:37:37,548 INFO:root:copying data_juicer/ops/grouper/__init__.py -> build/lib/data_juicer/ops/grouper 2025-03-28T21:37:37,550 INFO:root:creating build/lib/data_juicer/ops/selector 2025-03-28T21:37:37,551 INFO:root:copying data_juicer/ops/selector/topk_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-03-28T21:37:37,553 INFO:root:copying data_juicer/ops/selector/frequency_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-03-28T21:37:37,555 INFO:root:copying data_juicer/ops/selector/tags_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-03-28T21:37:37,556 INFO:root:copying data_juicer/ops/selector/random_selector.py -> build/lib/data_juicer/ops/selector 2025-03-28T21:37:37,558 INFO:root:copying data_juicer/ops/selector/range_specified_field_selector.py -> build/lib/data_juicer/ops/selector 2025-03-28T21:37:37,560 INFO:root:copying data_juicer/ops/selector/__init__.py -> build/lib/data_juicer/ops/selector 2025-03-28T21:37:37,564 INFO:root:creating build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,565 INFO:root:copying data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,567 INFO:root:copying data_juicer/ops/mapper/generate_qa_from_examples_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,570 INFO:root:copying data_juicer/ops/mapper/punctuation_normalization_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,571 INFO:root:copying data_juicer/ops/mapper/image_diffusion_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,573 INFO:root:copying data_juicer/ops/mapper/video_resize_resolution_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,575 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,578 INFO:root:copying data_juicer/ops/mapper/remove_non_chinese_character_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,580 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,582 INFO:root:copying data_juicer/ops/mapper/image_face_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,584 INFO:root:copying data_juicer/ops/mapper/calibrate_response_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,586 INFO:root:copying data_juicer/ops/mapper/query_intent_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,588 INFO:root:copying data_juicer/ops/mapper/text_chunk_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,590 INFO:root:copying data_juicer/ops/mapper/remove_long_words_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,592 INFO:root:copying data_juicer/ops/mapper/expand_macro_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,594 INFO:root:copying data_juicer/ops/mapper/image_segment_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,595 INFO:root:copying data_juicer/ops/mapper/whitespace_normalization_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,597 INFO:root:copying data_juicer/ops/mapper/dialog_sentiment_intensity_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,599 INFO:root:copying data_juicer/ops/mapper/optimize_response_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,601 INFO:root:copying data_juicer/ops/mapper/extract_entity_attribute_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,603 INFO:root:copying data_juicer/ops/mapper/fix_unicode_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,605 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_video_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,607 INFO:root:copying data_juicer/ops/mapper/extract_event_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,610 INFO:root:copying data_juicer/ops/mapper/image_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,611 INFO:root:copying data_juicer/ops/mapper/mllm_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,613 INFO:root:copying data_juicer/ops/mapper/clean_email_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,615 INFO:root:copying data_juicer/ops/mapper/audio_add_gaussian_noise_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,617 INFO:root:copying data_juicer/ops/mapper/pair_preference_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,619 INFO:root:copying data_juicer/ops/mapper/remove_repeat_sentences_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,621 INFO:root:copying data_juicer/ops/mapper/sentence_split_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,623 INFO:root:copying data_juicer/ops/mapper/query_sentiment_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,624 INFO:root:copying data_juicer/ops/mapper/sdxl_prompt2prompt_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,626 INFO:root:copying data_juicer/ops/mapper/python_lambda_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,628 INFO:root:copying data_juicer/ops/mapper/extract_entity_relation_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,631 INFO:root:copying data_juicer/ops/mapper/dialog_intent_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,633 INFO:root:copying data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,634 INFO:root:copying data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,636 INFO:root:copying data_juicer/ops/mapper/clean_links_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,638 INFO:root:copying data_juicer/ops/mapper/extract_support_text_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,640 INFO:root:copying data_juicer/ops/mapper/image_captioning_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,643 INFO:root:copying data_juicer/ops/mapper/image_tagging_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,645 INFO:root:copying data_juicer/ops/mapper/generate_qa_from_text_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,647 INFO:root:copying data_juicer/ops/mapper/query_topic_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,649 INFO:root:copying data_juicer/ops/mapper/video_split_by_scene_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,651 INFO:root:copying data_juicer/ops/mapper/remove_header_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,653 INFO:root:copying data_juicer/ops/mapper/chinese_convert_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,655 INFO:root:copying data_juicer/ops/mapper/video_extract_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,657 INFO:root:copying data_juicer/ops/mapper/extract_nickname_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,659 INFO:root:copying data_juicer/ops/mapper/optimize_qa_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,661 INFO:root:copying data_juicer/ops/mapper/sentence_augmentation_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,663 INFO:root:copying data_juicer/ops/mapper/video_tagging_from_audio_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,665 INFO:root:copying data_juicer/ops/mapper/clean_copyright_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,667 INFO:root:copying data_juicer/ops/mapper/nlpcda_zh_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,669 INFO:root:copying data_juicer/ops/mapper/clean_ip_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,671 INFO:root:copying data_juicer/ops/mapper/image_remove_background_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,673 INFO:root:copying data_juicer/ops/mapper/video_split_by_key_frame_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,676 INFO:root:copying data_juicer/ops/mapper/video_captioning_from_audio_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,678 INFO:root:copying data_juicer/ops/mapper/clean_html_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,680 INFO:root:copying data_juicer/ops/mapper/remove_specific_chars_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,682 INFO:root:copying data_juicer/ops/mapper/remove_bibliography_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,683 INFO:root:copying data_juicer/ops/mapper/dialog_sentiment_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,686 INFO:root:copying data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,688 INFO:root:copying data_juicer/ops/mapper/video_tagging_from_frames_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,690 INFO:root:copying data_juicer/ops/mapper/optimize_query_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,692 INFO:root:copying data_juicer/ops/mapper/__init__.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,694 INFO:root:copying data_juicer/ops/mapper/video_face_blur_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,696 INFO:root:copying data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,698 INFO:root:copying data_juicer/ops/mapper/remove_table_text_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,700 INFO:root:copying data_juicer/ops/mapper/calibrate_query_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,702 INFO:root:copying data_juicer/ops/mapper/dialog_topic_detection_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,704 INFO:root:copying data_juicer/ops/mapper/video_remove_watermark_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,706 INFO:root:copying data_juicer/ops/mapper/python_file_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,708 INFO:root:copying data_juicer/ops/mapper/replace_content_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,710 INFO:root:copying data_juicer/ops/mapper/extract_keyword_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,712 INFO:root:copying data_juicer/ops/mapper/calibrate_qa_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,714 INFO:root:copying data_juicer/ops/mapper/remove_comments_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,716 INFO:root:copying data_juicer/ops/mapper/video_split_by_duration_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,718 INFO:root:copying data_juicer/ops/mapper/relation_identity_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,721 INFO:root:copying data_juicer/ops/mapper/nlpaug_en_mapper.py -> build/lib/data_juicer/ops/mapper 2025-03-28T21:37:37,724 INFO:root:creating build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,725 INFO:root:copying data_juicer/ops/filter/video_watermark_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,727 INFO:root:copying data_juicer/ops/filter/video_motion_score_raft_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,729 INFO:root:copying data_juicer/ops/filter/image_nsfw_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,731 INFO:root:copying data_juicer/ops/filter/words_num_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,733 INFO:root:copying data_juicer/ops/filter/text_pair_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,735 INFO:root:copying data_juicer/ops/filter/video_duration_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,737 INFO:root:copying data_juicer/ops/filter/flagged_words_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,739 INFO:root:copying data_juicer/ops/filter/perplexity_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,741 INFO:root:copying data_juicer/ops/filter/word_repetition_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,743 INFO:root:copying data_juicer/ops/filter/phrase_grounding_recall_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,746 INFO:root:copying data_juicer/ops/filter/specified_field_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,747 INFO:root:copying data_juicer/ops/filter/video_aspect_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,749 INFO:root:copying data_juicer/ops/filter/character_repetition_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,751 INFO:root:copying data_juicer/ops/filter/image_text_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,753 INFO:root:copying data_juicer/ops/filter/image_face_count_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,755 INFO:root:copying data_juicer/ops/filter/image_face_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,757 INFO:root:copying data_juicer/ops/filter/video_nsfw_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,759 INFO:root:copying data_juicer/ops/filter/maximum_line_length_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,761 INFO:root:copying data_juicer/ops/filter/stopwords_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,763 INFO:root:copying data_juicer/ops/filter/video_motion_score_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,765 INFO:root:copying data_juicer/ops/filter/text_length_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,767 INFO:root:copying data_juicer/ops/filter/llm_quality_score_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,770 INFO:root:copying data_juicer/ops/filter/image_aspect_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,772 INFO:root:copying data_juicer/ops/filter/audio_size_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,773 INFO:root:copying data_juicer/ops/filter/suffix_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,775 INFO:root:copying data_juicer/ops/filter/token_num_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,777 INFO:root:copying data_juicer/ops/filter/video_aesthetics_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,779 INFO:root:copying data_juicer/ops/filter/video_ocr_area_ratio_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,781 INFO:root:copying data_juicer/ops/filter/audio_duration_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,783 INFO:root:copying data_juicer/ops/filter/image_shape_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,784 INFO:root:copying data_juicer/ops/filter/average_line_length_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,786 INFO:root:copying data_juicer/ops/filter/image_size_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,788 INFO:root:copying data_juicer/ops/filter/llm_difficulty_score_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,796 INFO:root:copying data_juicer/ops/filter/image_aesthetics_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,799 INFO:root:copying data_juicer/ops/filter/image_watermark_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,801 INFO:root:copying data_juicer/ops/filter/video_resolution_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,803 INFO:root:copying data_juicer/ops/filter/language_id_score_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,805 INFO:root:copying data_juicer/ops/filter/__init__.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,807 INFO:root:copying data_juicer/ops/filter/text_entity_dependency_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,808 INFO:root:copying data_juicer/ops/filter/image_pair_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,811 INFO:root:copying data_juicer/ops/filter/text_action_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,812 INFO:root:copying data_juicer/ops/filter/image_text_matching_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,814 INFO:root:copying data_juicer/ops/filter/video_frames_text_similarity_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,817 INFO:root:copying data_juicer/ops/filter/special_characters_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,819 INFO:root:copying data_juicer/ops/filter/video_tagging_from_frames_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,821 INFO:root:copying data_juicer/ops/filter/specified_numeric_field_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,823 INFO:root:copying data_juicer/ops/filter/alphanumeric_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,825 INFO:root:copying data_juicer/ops/filter/audio_nmf_snr_filter.py -> build/lib/data_juicer/ops/filter 2025-03-28T21:37:37,827 INFO:root:creating build/lib/data_juicer/ops/common 2025-03-28T21:37:37,829 INFO:root:copying data_juicer/ops/common/special_characters.py -> build/lib/data_juicer/ops/common 2025-03-28T21:37:37,831 INFO:root:copying data_juicer/ops/common/prompt2prompt_pipeline.py -> build/lib/data_juicer/ops/common 2025-03-28T21:37:37,834 INFO:root:copying data_juicer/ops/common/helper_func.py -> build/lib/data_juicer/ops/common 2025-03-28T21:37:37,836 INFO:root:copying data_juicer/ops/common/__init__.py -> build/lib/data_juicer/ops/common 2025-03-28T21:37:37,838 INFO:root:creating build/lib/data_juicer/ops/aggregator 2025-03-28T21:37:37,839 INFO:root:copying data_juicer/ops/aggregator/entity_attribute_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-03-28T21:37:37,842 INFO:root:copying data_juicer/ops/aggregator/meta_tags_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-03-28T21:37:37,844 INFO:root:copying data_juicer/ops/aggregator/most_relevant_entities_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-03-28T21:37:37,846 INFO:root:copying data_juicer/ops/aggregator/nested_aggregator.py -> build/lib/data_juicer/ops/aggregator 2025-03-28T21:37:37,848 INFO:root:copying data_juicer/ops/aggregator/__init__.py -> build/lib/data_juicer/ops/aggregator 2025-03-28T21:37:37,851 INFO:root:creating build/lib/data_juicer/core/executor 2025-03-28T21:37:37,852 INFO:root:copying data_juicer/core/executor/base.py -> build/lib/data_juicer/core/executor 2025-03-28T21:37:37,854 INFO:root:copying data_juicer/core/executor/default_executor.py -> build/lib/data_juicer/core/executor 2025-03-28T21:37:37,856 INFO:root:copying data_juicer/core/executor/factory.py -> build/lib/data_juicer/core/executor 2025-03-28T21:37:37,858 INFO:root:copying data_juicer/core/executor/ray_executor.py -> build/lib/data_juicer/core/executor 2025-03-28T21:37:37,860 INFO:root:copying data_juicer/core/executor/__init__.py -> build/lib/data_juicer/core/executor 2025-03-28T21:37:37,862 INFO:root:creating build/lib/data_juicer/core/data 2025-03-28T21:37:37,863 INFO:root:copying data_juicer/core/data/config_validator.py -> build/lib/data_juicer/core/data 2025-03-28T21:37:37,865 INFO:root:copying data_juicer/core/data/dataset_builder.py -> build/lib/data_juicer/core/data 2025-03-28T21:37:37,868 INFO:root:copying data_juicer/core/data/load_strategy.py -> build/lib/data_juicer/core/data 2025-03-28T21:37:37,870 INFO:root:copying data_juicer/core/data/data_validator.py -> build/lib/data_juicer/core/data 2025-03-28T21:37:37,872 INFO:root:copying data_juicer/core/data/schema.py -> build/lib/data_juicer/core/data 2025-03-28T21:37:37,874 INFO:root:copying data_juicer/core/data/dj_dataset.py -> build/lib/data_juicer/core/data 2025-03-28T21:37:37,876 INFO:root:copying data_juicer/core/data/__init__.py -> build/lib/data_juicer/core/data 2025-03-28T21:37:37,878 INFO:root:copying data_juicer/core/data/ray_dataset.py -> build/lib/data_juicer/core/data 2025-03-28T21:37:37,881 INFO:root:creating build/lib/data_juicer/tools 2025-03-28T21:37:37,882 INFO:root:copying tools/sandbox_starter.py -> build/lib/data_juicer/tools 2025-03-28T21:37:37,884 INFO:root:copying tools/dj_install.py -> build/lib/data_juicer/tools 2025-03-28T21:37:37,887 INFO:root:copying tools/analyze_data.py -> build/lib/data_juicer/tools 2025-03-28T21:37:37,889 INFO:root:copying tools/data_resplit.py -> build/lib/data_juicer/tools 2025-03-28T21:37:37,891 INFO:root:copying tools/__init__.py -> build/lib/data_juicer/tools 2025-03-28T21:37:37,892 INFO:root:copying tools/process_data.py -> build/lib/data_juicer/tools 2025-03-28T21:37:37,943 /usr/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. 2025-03-28T21:37:37,944 !! 2025-03-28T21:37:37,945 ******************************************************************************** 2025-03-28T21:37:37,945 Please avoid running ``setup.py`` directly. 2025-03-28T21:37:37,946 Instead, use pypa/build, pypa/installer or other 2025-03-28T21:37:37,946 standards-based tools. 2025-03-28T21:37:37,947 See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. 2025-03-28T21:37:37,948 ******************************************************************************** 2025-03-28T21:37:37,949 !! 2025-03-28T21:37:37,949 self.initialize_options() 2025-03-28T21:37:37,971 INFO:root:installing to build/bdist.linux-armv7l/wheel 2025-03-28T21:37:37,972 INFO:root:running install 2025-03-28T21:37:37,995 INFO:root:running install_lib 2025-03-28T21:37:38,022 INFO:root:creating build/bdist.linux-armv7l/wheel 2025-03-28T21:37:38,025 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer 2025-03-28T21:37:38,026 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/format 2025-03-28T21:37:38,028 INFO:root:copying build/lib/data_juicer/format/formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-03-28T21:37:38,030 INFO:root:copying build/lib/data_juicer/format/text_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-03-28T21:37:38,032 INFO:root:copying build/lib/data_juicer/format/parquet_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-03-28T21:37:38,034 INFO:root:copying build/lib/data_juicer/format/empty_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-03-28T21:37:38,036 INFO:root:copying build/lib/data_juicer/format/load.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-03-28T21:37:38,038 INFO:root:copying build/lib/data_juicer/format/tsv_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-03-28T21:37:38,039 INFO:root:copying build/lib/data_juicer/format/csv_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-03-28T21:37:38,041 INFO:root:copying build/lib/data_juicer/format/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-03-28T21:37:38,043 INFO:root:copying build/lib/data_juicer/format/json_formatter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/format 2025-03-28T21:37:38,045 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/analysis 2025-03-28T21:37:38,046 INFO:root:copying build/lib/data_juicer/analysis/collector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-03-28T21:37:38,048 INFO:root:copying build/lib/data_juicer/analysis/measure.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-03-28T21:37:38,050 INFO:root:copying build/lib/data_juicer/analysis/draw.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-03-28T21:37:38,052 INFO:root:copying build/lib/data_juicer/analysis/column_wise_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-03-28T21:37:38,055 INFO:root:copying build/lib/data_juicer/analysis/diversity_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-03-28T21:37:38,057 INFO:root:copying build/lib/data_juicer/analysis/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-03-28T21:37:38,059 INFO:root:copying build/lib/data_juicer/analysis/overall_analysis.py -> build/bdist.linux-armv7l/wheel/./data_juicer/analysis 2025-03-28T21:37:38,061 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/download 2025-03-28T21:37:38,062 INFO:root:copying build/lib/data_juicer/download/arxiv.py -> build/bdist.linux-armv7l/wheel/./data_juicer/download 2025-03-28T21:37:38,064 INFO:root:copying build/lib/data_juicer/download/commoncrawl.py -> build/bdist.linux-armv7l/wheel/./data_juicer/download 2025-03-28T21:37:38,066 INFO:root:copying build/lib/data_juicer/download/downloader.py -> build/bdist.linux-armv7l/wheel/./data_juicer/download 2025-03-28T21:37:38,068 INFO:root:copying build/lib/data_juicer/download/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/download 2025-03-28T21:37:38,070 INFO:root:copying build/lib/data_juicer/download/wikipedia.py -> build/bdist.linux-armv7l/wheel/./data_juicer/download 2025-03-28T21:37:38,073 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/tools 2025-03-28T21:37:38,074 INFO:root:copying build/lib/data_juicer/tools/sandbox_starter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-03-28T21:37:38,076 INFO:root:copying build/lib/data_juicer/tools/dj_install.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-03-28T21:37:38,077 INFO:root:copying build/lib/data_juicer/tools/analyze_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-03-28T21:37:38,079 INFO:root:copying build/lib/data_juicer/tools/data_resplit.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-03-28T21:37:38,081 INFO:root:copying build/lib/data_juicer/tools/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-03-28T21:37:38,083 INFO:root:copying build/lib/data_juicer/tools/process_data.py -> build/bdist.linux-armv7l/wheel/./data_juicer/tools 2025-03-28T21:37:38,085 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/config 2025-03-28T21:37:38,086 INFO:root:copying build/lib/data_juicer/config/config.py -> build/bdist.linux-armv7l/wheel/./data_juicer/config 2025-03-28T21:37:38,089 INFO:root:copying build/lib/data_juicer/config/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/config 2025-03-28T21:37:38,091 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/utils 2025-03-28T21:37:38,093 INFO:root:copying build/lib/data_juicer/utils/compress.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,095 INFO:root:copying build/lib/data_juicer/utils/logger_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,098 INFO:root:copying build/lib/data_juicer/utils/model_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,101 INFO:root:copying build/lib/data_juicer/utils/resource_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,102 INFO:root:copying build/lib/data_juicer/utils/auto_install_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,105 INFO:root:copying build/lib/data_juicer/utils/process_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,107 INFO:root:copying build/lib/data_juicer/utils/availability_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,109 INFO:root:copying build/lib/data_juicer/utils/unittest_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,111 INFO:root:copying build/lib/data_juicer/utils/asset_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,113 INFO:root:copying build/lib/data_juicer/utils/cache_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,115 INFO:root:copying build/lib/data_juicer/utils/mm_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,118 INFO:root:copying build/lib/data_juicer/utils/constant.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,120 INFO:root:copying build/lib/data_juicer/utils/auto_install_mapping.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,123 INFO:root:copying build/lib/data_juicer/utils/file_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,125 INFO:root:copying build/lib/data_juicer/utils/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,127 INFO:root:copying build/lib/data_juicer/utils/lazy_loader.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,129 INFO:root:copying build/lib/data_juicer/utils/sample.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,131 INFO:root:copying build/lib/data_juicer/utils/common_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,133 INFO:root:copying build/lib/data_juicer/utils/ckpt_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,135 INFO:root:copying build/lib/data_juicer/utils/registry.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,137 INFO:root:copying build/lib/data_juicer/utils/fingerprint_utils.py -> build/bdist.linux-armv7l/wheel/./data_juicer/utils 2025-03-28T21:37:38,140 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops 2025-03-28T21:37:38,142 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/deduplicator 2025-03-28T21:37:38,143 INFO:root:copying build/lib/data_juicer/ops/deduplicator/video_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,146 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_basic_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,147 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_image_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,150 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_minhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,152 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,154 INFO:root:copying build/lib/data_juicer/ops/deduplicator/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,156 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_bts_minhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,158 INFO:root:copying build/lib/data_juicer/ops/deduplicator/document_simhash_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,161 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_video_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,163 INFO:root:copying build/lib/data_juicer/ops/deduplicator/image_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,165 INFO:root:copying build/lib/data_juicer/ops/deduplicator/ray_document_deduplicator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/deduplicator 2025-03-28T21:37:38,167 INFO:root:copying build/lib/data_juicer/ops/base_op.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-03-28T21:37:38,171 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/grouper 2025-03-28T21:37:38,172 INFO:root:copying build/lib/data_juicer/ops/grouper/key_value_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-03-28T21:37:38,174 INFO:root:copying build/lib/data_juicer/ops/grouper/naive_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-03-28T21:37:38,176 INFO:root:copying build/lib/data_juicer/ops/grouper/naive_reverse_grouper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-03-28T21:37:38,178 INFO:root:copying build/lib/data_juicer/ops/grouper/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/grouper 2025-03-28T21:37:38,181 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/selector 2025-03-28T21:37:38,182 INFO:root:copying build/lib/data_juicer/ops/selector/topk_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-03-28T21:37:38,184 INFO:root:copying build/lib/data_juicer/ops/selector/frequency_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-03-28T21:37:38,186 INFO:root:copying build/lib/data_juicer/ops/selector/tags_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-03-28T21:37:38,188 INFO:root:copying build/lib/data_juicer/ops/selector/random_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-03-28T21:37:38,190 INFO:root:copying build/lib/data_juicer/ops/selector/range_specified_field_selector.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-03-28T21:37:38,192 INFO:root:copying build/lib/data_juicer/ops/selector/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/selector 2025-03-28T21:37:38,195 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/mapper 2025-03-28T21:37:38,196 INFO:root:copying build/lib/data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,199 INFO:root:copying build/lib/data_juicer/ops/mapper/generate_qa_from_examples_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,201 INFO:root:copying build/lib/data_juicer/ops/mapper/punctuation_normalization_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,203 INFO:root:copying build/lib/data_juicer/ops/mapper/image_diffusion_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,205 INFO:root:copying build/lib/data_juicer/ops/mapper/video_resize_resolution_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,207 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,210 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_non_chinese_character_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,212 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,214 INFO:root:copying build/lib/data_juicer/ops/mapper/image_face_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,216 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_response_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,218 INFO:root:copying build/lib/data_juicer/ops/mapper/query_intent_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,220 INFO:root:copying build/lib/data_juicer/ops/mapper/text_chunk_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,222 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_long_words_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,224 INFO:root:copying build/lib/data_juicer/ops/mapper/expand_macro_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,226 INFO:root:copying build/lib/data_juicer/ops/mapper/image_segment_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,228 INFO:root:copying build/lib/data_juicer/ops/mapper/whitespace_normalization_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,230 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_sentiment_intensity_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,232 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_response_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,234 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_entity_attribute_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,236 INFO:root:copying build/lib/data_juicer/ops/mapper/fix_unicode_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,238 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_video_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,240 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_event_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,242 INFO:root:copying build/lib/data_juicer/ops/mapper/image_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,244 INFO:root:copying build/lib/data_juicer/ops/mapper/mllm_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,246 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_email_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,248 INFO:root:copying build/lib/data_juicer/ops/mapper/audio_add_gaussian_noise_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,250 INFO:root:copying build/lib/data_juicer/ops/mapper/pair_preference_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,252 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_repeat_sentences_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,254 INFO:root:copying build/lib/data_juicer/ops/mapper/sentence_split_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,255 INFO:root:copying build/lib/data_juicer/ops/mapper/query_sentiment_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,257 INFO:root:copying build/lib/data_juicer/ops/mapper/sdxl_prompt2prompt_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,259 INFO:root:copying build/lib/data_juicer/ops/mapper/python_lambda_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,261 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_entity_relation_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,263 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_intent_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,265 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,267 INFO:root:copying build/lib/data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,269 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_links_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,271 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_support_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,273 INFO:root:copying build/lib/data_juicer/ops/mapper/image_captioning_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,275 INFO:root:copying build/lib/data_juicer/ops/mapper/image_tagging_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,277 INFO:root:copying build/lib/data_juicer/ops/mapper/generate_qa_from_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,280 INFO:root:copying build/lib/data_juicer/ops/mapper/query_topic_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,281 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_scene_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,283 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_header_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,285 INFO:root:copying build/lib/data_juicer/ops/mapper/chinese_convert_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,287 INFO:root:copying build/lib/data_juicer/ops/mapper/video_extract_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,289 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_nickname_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,291 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_qa_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,293 INFO:root:copying build/lib/data_juicer/ops/mapper/sentence_augmentation_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,295 INFO:root:copying build/lib/data_juicer/ops/mapper/video_tagging_from_audio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,297 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_copyright_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,299 INFO:root:copying build/lib/data_juicer/ops/mapper/nlpcda_zh_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,301 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_ip_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,302 INFO:root:copying build/lib/data_juicer/ops/mapper/image_remove_background_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,304 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_key_frame_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,306 INFO:root:copying build/lib/data_juicer/ops/mapper/video_captioning_from_audio_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,309 INFO:root:copying build/lib/data_juicer/ops/mapper/clean_html_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,310 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_specific_chars_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,312 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_bibliography_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,314 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_sentiment_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,316 INFO:root:copying build/lib/data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,319 INFO:root:copying build/lib/data_juicer/ops/mapper/video_tagging_from_frames_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,321 INFO:root:copying build/lib/data_juicer/ops/mapper/optimize_query_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,323 INFO:root:copying build/lib/data_juicer/ops/mapper/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,325 INFO:root:copying build/lib/data_juicer/ops/mapper/video_face_blur_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,327 INFO:root:copying build/lib/data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,329 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_table_text_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,331 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_query_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,332 INFO:root:copying build/lib/data_juicer/ops/mapper/dialog_topic_detection_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,335 INFO:root:copying build/lib/data_juicer/ops/mapper/video_remove_watermark_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,337 INFO:root:copying build/lib/data_juicer/ops/mapper/python_file_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,338 INFO:root:copying build/lib/data_juicer/ops/mapper/replace_content_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,340 INFO:root:copying build/lib/data_juicer/ops/mapper/extract_keyword_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,342 INFO:root:copying build/lib/data_juicer/ops/mapper/calibrate_qa_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,344 INFO:root:copying build/lib/data_juicer/ops/mapper/remove_comments_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,346 INFO:root:copying build/lib/data_juicer/ops/mapper/video_split_by_duration_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,349 INFO:root:copying build/lib/data_juicer/ops/mapper/relation_identity_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,351 INFO:root:copying build/lib/data_juicer/ops/mapper/nlpaug_en_mapper.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/mapper 2025-03-28T21:37:38,353 INFO:root:copying build/lib/data_juicer/ops/load.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-03-28T21:37:38,356 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/filter 2025-03-28T21:37:38,357 INFO:root:copying build/lib/data_juicer/ops/filter/video_watermark_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,359 INFO:root:copying build/lib/data_juicer/ops/filter/video_motion_score_raft_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,361 INFO:root:copying build/lib/data_juicer/ops/filter/image_nsfw_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,363 INFO:root:copying build/lib/data_juicer/ops/filter/words_num_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,365 INFO:root:copying build/lib/data_juicer/ops/filter/text_pair_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,367 INFO:root:copying build/lib/data_juicer/ops/filter/video_duration_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,369 INFO:root:copying build/lib/data_juicer/ops/filter/flagged_words_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,371 INFO:root:copying build/lib/data_juicer/ops/filter/perplexity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,373 INFO:root:copying build/lib/data_juicer/ops/filter/word_repetition_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,375 INFO:root:copying build/lib/data_juicer/ops/filter/phrase_grounding_recall_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,377 INFO:root:copying build/lib/data_juicer/ops/filter/specified_field_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,379 INFO:root:copying build/lib/data_juicer/ops/filter/video_aspect_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,381 INFO:root:copying build/lib/data_juicer/ops/filter/character_repetition_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,383 INFO:root:copying build/lib/data_juicer/ops/filter/image_text_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,385 INFO:root:copying build/lib/data_juicer/ops/filter/image_face_count_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,387 INFO:root:copying build/lib/data_juicer/ops/filter/image_face_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,389 INFO:root:copying build/lib/data_juicer/ops/filter/video_nsfw_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,391 INFO:root:copying build/lib/data_juicer/ops/filter/maximum_line_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,393 INFO:root:copying build/lib/data_juicer/ops/filter/stopwords_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,395 INFO:root:copying build/lib/data_juicer/ops/filter/video_motion_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,397 INFO:root:copying build/lib/data_juicer/ops/filter/text_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,399 INFO:root:copying build/lib/data_juicer/ops/filter/llm_quality_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,401 INFO:root:copying build/lib/data_juicer/ops/filter/image_aspect_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,403 INFO:root:copying build/lib/data_juicer/ops/filter/audio_size_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,405 INFO:root:copying build/lib/data_juicer/ops/filter/suffix_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,407 INFO:root:copying build/lib/data_juicer/ops/filter/token_num_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,409 INFO:root:copying build/lib/data_juicer/ops/filter/video_aesthetics_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,412 INFO:root:copying build/lib/data_juicer/ops/filter/video_ocr_area_ratio_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,414 INFO:root:copying build/lib/data_juicer/ops/filter/audio_duration_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,416 INFO:root:copying build/lib/data_juicer/ops/filter/image_shape_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,418 INFO:root:copying build/lib/data_juicer/ops/filter/average_line_length_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,420 INFO:root:copying build/lib/data_juicer/ops/filter/image_size_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,421 INFO:root:copying build/lib/data_juicer/ops/filter/llm_difficulty_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,424 INFO:root:copying build/lib/data_juicer/ops/filter/image_aesthetics_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,426 INFO:root:copying build/lib/data_juicer/ops/filter/image_watermark_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,428 INFO:root:copying build/lib/data_juicer/ops/filter/video_resolution_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,430 INFO:root:copying build/lib/data_juicer/ops/filter/language_id_score_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,431 INFO:root:copying build/lib/data_juicer/ops/filter/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,434 INFO:root:copying build/lib/data_juicer/ops/filter/text_entity_dependency_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,435 INFO:root:copying build/lib/data_juicer/ops/filter/image_pair_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,437 INFO:root:copying build/lib/data_juicer/ops/filter/text_action_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,439 INFO:root:copying build/lib/data_juicer/ops/filter/image_text_matching_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,441 INFO:root:copying build/lib/data_juicer/ops/filter/video_frames_text_similarity_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,443 INFO:root:copying build/lib/data_juicer/ops/filter/special_characters_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,445 INFO:root:copying build/lib/data_juicer/ops/filter/video_tagging_from_frames_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,448 INFO:root:copying build/lib/data_juicer/ops/filter/specified_numeric_field_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,450 INFO:root:copying build/lib/data_juicer/ops/filter/alphanumeric_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,451 INFO:root:copying build/lib/data_juicer/ops/filter/audio_nmf_snr_filter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/filter 2025-03-28T21:37:38,454 INFO:root:copying build/lib/data_juicer/ops/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-03-28T21:37:38,455 INFO:root:copying build/lib/data_juicer/ops/op_fusion.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops 2025-03-28T21:37:38,458 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/common 2025-03-28T21:37:38,459 INFO:root:copying build/lib/data_juicer/ops/common/special_characters.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2025-03-28T21:37:38,461 INFO:root:copying build/lib/data_juicer/ops/common/prompt2prompt_pipeline.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2025-03-28T21:37:38,464 INFO:root:copying build/lib/data_juicer/ops/common/helper_func.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2025-03-28T21:37:38,466 INFO:root:copying build/lib/data_juicer/ops/common/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/common 2025-03-28T21:37:38,469 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/ops/aggregator 2025-03-28T21:37:38,470 INFO:root:copying build/lib/data_juicer/ops/aggregator/entity_attribute_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-03-28T21:37:38,472 INFO:root:copying build/lib/data_juicer/ops/aggregator/meta_tags_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-03-28T21:37:38,475 INFO:root:copying build/lib/data_juicer/ops/aggregator/most_relevant_entities_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-03-28T21:37:38,477 INFO:root:copying build/lib/data_juicer/ops/aggregator/nested_aggregator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-03-28T21:37:38,480 INFO:root:copying build/lib/data_juicer/ops/aggregator/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/ops/aggregator 2025-03-28T21:37:38,482 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/core 2025-03-28T21:37:38,483 INFO:root:copying build/lib/data_juicer/core/tracer.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-03-28T21:37:38,486 INFO:root:copying build/lib/data_juicer/core/exporter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-03-28T21:37:38,489 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/core/executor 2025-03-28T21:37:38,490 INFO:root:copying build/lib/data_juicer/core/executor/base.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/executor 2025-03-28T21:37:38,492 INFO:root:copying build/lib/data_juicer/core/executor/default_executor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/executor 2025-03-28T21:37:38,494 INFO:root:copying build/lib/data_juicer/core/executor/factory.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/executor 2025-03-28T21:37:38,496 INFO:root:copying build/lib/data_juicer/core/executor/ray_executor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/executor 2025-03-28T21:37:38,498 INFO:root:copying build/lib/data_juicer/core/executor/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/executor 2025-03-28T21:37:38,499 INFO:root:copying build/lib/data_juicer/core/analyzer.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-03-28T21:37:38,502 INFO:root:creating build/bdist.linux-armv7l/wheel/data_juicer/core/data 2025-03-28T21:37:38,503 INFO:root:copying build/lib/data_juicer/core/data/config_validator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/data 2025-03-28T21:37:38,505 INFO:root:copying build/lib/data_juicer/core/data/dataset_builder.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/data 2025-03-28T21:37:38,507 INFO:root:copying build/lib/data_juicer/core/data/load_strategy.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/data 2025-03-28T21:37:38,509 INFO:root:copying build/lib/data_juicer/core/data/data_validator.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/data 2025-03-28T21:37:38,512 INFO:root:copying build/lib/data_juicer/core/data/schema.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/data 2025-03-28T21:37:38,513 INFO:root:copying build/lib/data_juicer/core/data/dj_dataset.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/data 2025-03-28T21:37:38,516 INFO:root:copying build/lib/data_juicer/core/data/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/data 2025-03-28T21:37:38,518 INFO:root:copying build/lib/data_juicer/core/data/ray_dataset.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core/data 2025-03-28T21:37:38,520 INFO:root:copying build/lib/data_juicer/core/monitor.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-03-28T21:37:38,522 INFO:root:copying build/lib/data_juicer/core/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-03-28T21:37:38,524 INFO:root:copying build/lib/data_juicer/core/adapter.py -> build/bdist.linux-armv7l/wheel/./data_juicer/core 2025-03-28T21:37:38,526 INFO:root:copying build/lib/data_juicer/__init__.py -> build/bdist.linux-armv7l/wheel/./data_juicer 2025-03-28T21:37:38,528 INFO:root:running install_egg_info 2025-03-28T21:37:38,562 INFO:root:running egg_info 2025-03-28T21:37:38,589 INFO:root:writing py_data_juicer.egg-info/PKG-INFO 2025-03-28T21:37:38,594 INFO:root:writing dependency_links to py_data_juicer.egg-info/dependency_links.txt 2025-03-28T21:37:38,595 INFO:root:writing entry points to py_data_juicer.egg-info/entry_points.txt 2025-03-28T21:37:38,597 INFO:root:writing requirements to py_data_juicer.egg-info/requires.txt 2025-03-28T21:37:38,598 INFO:root:writing top-level names to py_data_juicer.egg-info/top_level.txt 2025-03-28T21:37:38,648 INFO:root:reading manifest file 'py_data_juicer.egg-info/SOURCES.txt' 2025-03-28T21:37:38,665 INFO:root:adding license file 'LICENSE' 2025-03-28T21:37:38,677 INFO:root:writing manifest file 'py_data_juicer.egg-info/SOURCES.txt' 2025-03-28T21:37:38,679 INFO:root:Copying py_data_juicer.egg-info to build/bdist.linux-armv7l/wheel/./py_data_juicer-1.3.0-py3.11.egg-info 2025-03-28T21:37:38,692 INFO:root:running install_scripts 2025-03-28T21:37:38,712 INFO:root:creating build/bdist.linux-armv7l/wheel/py_data_juicer-1.3.0.dist-info/WHEEL 2025-03-28T21:37:38,715 INFO:wheel:creating '/tmp/pip-wheel-6xj1jnpg/py_data_juicer-1.3.0-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2025-03-28T21:37:38,717 INFO:wheel:adding 'data_juicer/__init__.py' 2025-03-28T21:37:38,719 INFO:wheel:adding 'data_juicer/analysis/__init__.py' 2025-03-28T21:37:38,720 INFO:wheel:adding 'data_juicer/analysis/collector.py' 2025-03-28T21:37:38,722 INFO:wheel:adding 'data_juicer/analysis/column_wise_analysis.py' 2025-03-28T21:37:38,724 INFO:wheel:adding 'data_juicer/analysis/diversity_analysis.py' 2025-03-28T21:37:38,725 INFO:wheel:adding 'data_juicer/analysis/draw.py' 2025-03-28T21:37:38,726 INFO:wheel:adding 'data_juicer/analysis/measure.py' 2025-03-28T21:37:38,728 INFO:wheel:adding 'data_juicer/analysis/overall_analysis.py' 2025-03-28T21:37:38,730 INFO:wheel:adding 'data_juicer/config/__init__.py' 2025-03-28T21:37:38,734 INFO:wheel:adding 'data_juicer/config/config.py' 2025-03-28T21:37:38,736 INFO:wheel:adding 'data_juicer/core/__init__.py' 2025-03-28T21:37:38,737 INFO:wheel:adding 'data_juicer/core/adapter.py' 2025-03-28T21:37:38,739 INFO:wheel:adding 'data_juicer/core/analyzer.py' 2025-03-28T21:37:38,741 INFO:wheel:adding 'data_juicer/core/exporter.py' 2025-03-28T21:37:38,743 INFO:wheel:adding 'data_juicer/core/monitor.py' 2025-03-28T21:37:38,744 INFO:wheel:adding 'data_juicer/core/tracer.py' 2025-03-28T21:37:38,746 INFO:wheel:adding 'data_juicer/core/data/__init__.py' 2025-03-28T21:37:38,747 INFO:wheel:adding 'data_juicer/core/data/config_validator.py' 2025-03-28T21:37:38,749 INFO:wheel:adding 'data_juicer/core/data/data_validator.py' 2025-03-28T21:37:38,751 INFO:wheel:adding 'data_juicer/core/data/dataset_builder.py' 2025-03-28T21:37:38,754 INFO:wheel:adding 'data_juicer/core/data/dj_dataset.py' 2025-03-28T21:37:38,756 INFO:wheel:adding 'data_juicer/core/data/load_strategy.py' 2025-03-28T21:37:38,758 INFO:wheel:adding 'data_juicer/core/data/ray_dataset.py' 2025-03-28T21:37:38,760 INFO:wheel:adding 'data_juicer/core/data/schema.py' 2025-03-28T21:37:38,761 INFO:wheel:adding 'data_juicer/core/executor/__init__.py' 2025-03-28T21:37:38,763 INFO:wheel:adding 'data_juicer/core/executor/base.py' 2025-03-28T21:37:38,764 INFO:wheel:adding 'data_juicer/core/executor/default_executor.py' 2025-03-28T21:37:38,766 INFO:wheel:adding 'data_juicer/core/executor/factory.py' 2025-03-28T21:37:38,767 INFO:wheel:adding 'data_juicer/core/executor/ray_executor.py' 2025-03-28T21:37:38,769 INFO:wheel:adding 'data_juicer/download/__init__.py' 2025-03-28T21:37:38,771 INFO:wheel:adding 'data_juicer/download/arxiv.py' 2025-03-28T21:37:38,772 INFO:wheel:adding 'data_juicer/download/commoncrawl.py' 2025-03-28T21:37:38,774 INFO:wheel:adding 'data_juicer/download/downloader.py' 2025-03-28T21:37:38,778 INFO:wheel:adding 'data_juicer/download/wikipedia.py' 2025-03-28T21:37:38,780 INFO:wheel:adding 'data_juicer/format/__init__.py' 2025-03-28T21:37:38,781 INFO:wheel:adding 'data_juicer/format/csv_formatter.py' 2025-03-28T21:37:38,782 INFO:wheel:adding 'data_juicer/format/empty_formatter.py' 2025-03-28T21:37:38,784 INFO:wheel:adding 'data_juicer/format/formatter.py' 2025-03-28T21:37:38,785 INFO:wheel:adding 'data_juicer/format/json_formatter.py' 2025-03-28T21:37:38,786 INFO:wheel:adding 'data_juicer/format/load.py' 2025-03-28T21:37:38,787 INFO:wheel:adding 'data_juicer/format/parquet_formatter.py' 2025-03-28T21:37:38,789 INFO:wheel:adding 'data_juicer/format/text_formatter.py' 2025-03-28T21:37:38,790 INFO:wheel:adding 'data_juicer/format/tsv_formatter.py' 2025-03-28T21:37:38,793 INFO:wheel:adding 'data_juicer/ops/__init__.py' 2025-03-28T21:37:38,795 INFO:wheel:adding 'data_juicer/ops/base_op.py' 2025-03-28T21:37:38,796 INFO:wheel:adding 'data_juicer/ops/load.py' 2025-03-28T21:37:38,798 INFO:wheel:adding 'data_juicer/ops/op_fusion.py' 2025-03-28T21:37:38,800 INFO:wheel:adding 'data_juicer/ops/aggregator/__init__.py' 2025-03-28T21:37:38,802 INFO:wheel:adding 'data_juicer/ops/aggregator/entity_attribute_aggregator.py' 2025-03-28T21:37:38,804 INFO:wheel:adding 'data_juicer/ops/aggregator/meta_tags_aggregator.py' 2025-03-28T21:37:38,805 INFO:wheel:adding 'data_juicer/ops/aggregator/most_relevant_entities_aggregator.py' 2025-03-28T21:37:38,807 INFO:wheel:adding 'data_juicer/ops/aggregator/nested_aggregator.py' 2025-03-28T21:37:38,808 INFO:wheel:adding 'data_juicer/ops/common/__init__.py' 2025-03-28T21:37:38,810 INFO:wheel:adding 'data_juicer/ops/common/helper_func.py' 2025-03-28T21:37:38,815 INFO:wheel:adding 'data_juicer/ops/common/prompt2prompt_pipeline.py' 2025-03-28T21:37:38,817 INFO:wheel:adding 'data_juicer/ops/common/special_characters.py' 2025-03-28T21:37:38,818 INFO:wheel:adding 'data_juicer/ops/deduplicator/__init__.py' 2025-03-28T21:37:38,820 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_deduplicator.py' 2025-03-28T21:37:38,821 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_minhash_deduplicator.py' 2025-03-28T21:37:38,823 INFO:wheel:adding 'data_juicer/ops/deduplicator/document_simhash_deduplicator.py' 2025-03-28T21:37:38,825 INFO:wheel:adding 'data_juicer/ops/deduplicator/image_deduplicator.py' 2025-03-28T21:37:38,826 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_basic_deduplicator.py' 2025-03-28T21:37:38,829 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_bts_minhash_deduplicator.py' 2025-03-28T21:37:38,830 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_document_deduplicator.py' 2025-03-28T21:37:38,832 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_image_deduplicator.py' 2025-03-28T21:37:38,833 INFO:wheel:adding 'data_juicer/ops/deduplicator/ray_video_deduplicator.py' 2025-03-28T21:37:38,834 INFO:wheel:adding 'data_juicer/ops/deduplicator/video_deduplicator.py' 2025-03-28T21:37:38,837 INFO:wheel:adding 'data_juicer/ops/filter/__init__.py' 2025-03-28T21:37:38,838 INFO:wheel:adding 'data_juicer/ops/filter/alphanumeric_filter.py' 2025-03-28T21:37:38,840 INFO:wheel:adding 'data_juicer/ops/filter/audio_duration_filter.py' 2025-03-28T21:37:38,841 INFO:wheel:adding 'data_juicer/ops/filter/audio_nmf_snr_filter.py' 2025-03-28T21:37:38,842 INFO:wheel:adding 'data_juicer/ops/filter/audio_size_filter.py' 2025-03-28T21:37:38,844 INFO:wheel:adding 'data_juicer/ops/filter/average_line_length_filter.py' 2025-03-28T21:37:38,845 INFO:wheel:adding 'data_juicer/ops/filter/character_repetition_filter.py' 2025-03-28T21:37:38,846 INFO:wheel:adding 'data_juicer/ops/filter/flagged_words_filter.py' 2025-03-28T21:37:38,848 INFO:wheel:adding 'data_juicer/ops/filter/image_aesthetics_filter.py' 2025-03-28T21:37:38,849 INFO:wheel:adding 'data_juicer/ops/filter/image_aspect_ratio_filter.py' 2025-03-28T21:37:38,851 INFO:wheel:adding 'data_juicer/ops/filter/image_face_count_filter.py' 2025-03-28T21:37:38,852 INFO:wheel:adding 'data_juicer/ops/filter/image_face_ratio_filter.py' 2025-03-28T21:37:38,854 INFO:wheel:adding 'data_juicer/ops/filter/image_nsfw_filter.py' 2025-03-28T21:37:38,855 INFO:wheel:adding 'data_juicer/ops/filter/image_pair_similarity_filter.py' 2025-03-28T21:37:38,857 INFO:wheel:adding 'data_juicer/ops/filter/image_shape_filter.py' 2025-03-28T21:37:38,858 INFO:wheel:adding 'data_juicer/ops/filter/image_size_filter.py' 2025-03-28T21:37:38,860 INFO:wheel:adding 'data_juicer/ops/filter/image_text_matching_filter.py' 2025-03-28T21:37:38,861 INFO:wheel:adding 'data_juicer/ops/filter/image_text_similarity_filter.py' 2025-03-28T21:37:38,862 INFO:wheel:adding 'data_juicer/ops/filter/image_watermark_filter.py' 2025-03-28T21:37:38,864 INFO:wheel:adding 'data_juicer/ops/filter/language_id_score_filter.py' 2025-03-28T21:37:38,865 INFO:wheel:adding 'data_juicer/ops/filter/llm_difficulty_score_filter.py' 2025-03-28T21:37:38,867 INFO:wheel:adding 'data_juicer/ops/filter/llm_quality_score_filter.py' 2025-03-28T21:37:38,868 INFO:wheel:adding 'data_juicer/ops/filter/maximum_line_length_filter.py' 2025-03-28T21:37:38,870 INFO:wheel:adding 'data_juicer/ops/filter/perplexity_filter.py' 2025-03-28T21:37:38,872 INFO:wheel:adding 'data_juicer/ops/filter/phrase_grounding_recall_filter.py' 2025-03-28T21:37:38,873 INFO:wheel:adding 'data_juicer/ops/filter/special_characters_filter.py' 2025-03-28T21:37:38,874 INFO:wheel:adding 'data_juicer/ops/filter/specified_field_filter.py' 2025-03-28T21:37:38,875 INFO:wheel:adding 'data_juicer/ops/filter/specified_numeric_field_filter.py' 2025-03-28T21:37:38,876 INFO:wheel:adding 'data_juicer/ops/filter/stopwords_filter.py' 2025-03-28T21:37:38,878 INFO:wheel:adding 'data_juicer/ops/filter/suffix_filter.py' 2025-03-28T21:37:38,879 INFO:wheel:adding 'data_juicer/ops/filter/text_action_filter.py' 2025-03-28T21:37:38,880 INFO:wheel:adding 'data_juicer/ops/filter/text_entity_dependency_filter.py' 2025-03-28T21:37:38,882 INFO:wheel:adding 'data_juicer/ops/filter/text_length_filter.py' 2025-03-28T21:37:38,883 INFO:wheel:adding 'data_juicer/ops/filter/text_pair_similarity_filter.py' 2025-03-28T21:37:38,884 INFO:wheel:adding 'data_juicer/ops/filter/token_num_filter.py' 2025-03-28T21:37:38,886 INFO:wheel:adding 'data_juicer/ops/filter/video_aesthetics_filter.py' 2025-03-28T21:37:38,887 INFO:wheel:adding 'data_juicer/ops/filter/video_aspect_ratio_filter.py' 2025-03-28T21:37:38,889 INFO:wheel:adding 'data_juicer/ops/filter/video_duration_filter.py' 2025-03-28T21:37:38,890 INFO:wheel:adding 'data_juicer/ops/filter/video_frames_text_similarity_filter.py' 2025-03-28T21:37:38,892 INFO:wheel:adding 'data_juicer/ops/filter/video_motion_score_filter.py' 2025-03-28T21:37:38,894 INFO:wheel:adding 'data_juicer/ops/filter/video_motion_score_raft_filter.py' 2025-03-28T21:37:38,895 INFO:wheel:adding 'data_juicer/ops/filter/video_nsfw_filter.py' 2025-03-28T21:37:38,897 INFO:wheel:adding 'data_juicer/ops/filter/video_ocr_area_ratio_filter.py' 2025-03-28T21:37:38,899 INFO:wheel:adding 'data_juicer/ops/filter/video_resolution_filter.py' 2025-03-28T21:37:38,900 INFO:wheel:adding 'data_juicer/ops/filter/video_tagging_from_frames_filter.py' 2025-03-28T21:37:38,901 INFO:wheel:adding 'data_juicer/ops/filter/video_watermark_filter.py' 2025-03-28T21:37:38,903 INFO:wheel:adding 'data_juicer/ops/filter/word_repetition_filter.py' 2025-03-28T21:37:38,904 INFO:wheel:adding 'data_juicer/ops/filter/words_num_filter.py' 2025-03-28T21:37:38,906 INFO:wheel:adding 'data_juicer/ops/grouper/__init__.py' 2025-03-28T21:37:38,907 INFO:wheel:adding 'data_juicer/ops/grouper/key_value_grouper.py' 2025-03-28T21:37:38,908 INFO:wheel:adding 'data_juicer/ops/grouper/naive_grouper.py' 2025-03-28T21:37:38,910 INFO:wheel:adding 'data_juicer/ops/grouper/naive_reverse_grouper.py' 2025-03-28T21:37:38,913 INFO:wheel:adding 'data_juicer/ops/mapper/__init__.py' 2025-03-28T21:37:38,914 INFO:wheel:adding 'data_juicer/ops/mapper/audio_add_gaussian_noise_mapper.py' 2025-03-28T21:37:38,916 INFO:wheel:adding 'data_juicer/ops/mapper/audio_ffmpeg_wrapped_mapper.py' 2025-03-28T21:37:38,917 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_qa_mapper.py' 2025-03-28T21:37:38,919 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_query_mapper.py' 2025-03-28T21:37:38,920 INFO:wheel:adding 'data_juicer/ops/mapper/calibrate_response_mapper.py' 2025-03-28T21:37:38,922 INFO:wheel:adding 'data_juicer/ops/mapper/chinese_convert_mapper.py' 2025-03-28T21:37:38,923 INFO:wheel:adding 'data_juicer/ops/mapper/clean_copyright_mapper.py' 2025-03-28T21:37:38,925 INFO:wheel:adding 'data_juicer/ops/mapper/clean_email_mapper.py' 2025-03-28T21:37:38,926 INFO:wheel:adding 'data_juicer/ops/mapper/clean_html_mapper.py' 2025-03-28T21:37:38,927 INFO:wheel:adding 'data_juicer/ops/mapper/clean_ip_mapper.py' 2025-03-28T21:37:38,928 INFO:wheel:adding 'data_juicer/ops/mapper/clean_links_mapper.py' 2025-03-28T21:37:38,930 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_intent_detection_mapper.py' 2025-03-28T21:37:38,932 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_sentiment_detection_mapper.py' 2025-03-28T21:37:38,934 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_sentiment_intensity_mapper.py' 2025-03-28T21:37:38,936 INFO:wheel:adding 'data_juicer/ops/mapper/dialog_topic_detection_mapper.py' 2025-03-28T21:37:38,937 INFO:wheel:adding 'data_juicer/ops/mapper/expand_macro_mapper.py' 2025-03-28T21:37:38,939 INFO:wheel:adding 'data_juicer/ops/mapper/extract_entity_attribute_mapper.py' 2025-03-28T21:37:38,941 INFO:wheel:adding 'data_juicer/ops/mapper/extract_entity_relation_mapper.py' 2025-03-28T21:37:38,943 INFO:wheel:adding 'data_juicer/ops/mapper/extract_event_mapper.py' 2025-03-28T21:37:38,945 INFO:wheel:adding 'data_juicer/ops/mapper/extract_keyword_mapper.py' 2025-03-28T21:37:38,946 INFO:wheel:adding 'data_juicer/ops/mapper/extract_nickname_mapper.py' 2025-03-28T21:37:38,948 INFO:wheel:adding 'data_juicer/ops/mapper/extract_support_text_mapper.py' 2025-03-28T21:37:38,949 INFO:wheel:adding 'data_juicer/ops/mapper/fix_unicode_mapper.py' 2025-03-28T21:37:38,951 INFO:wheel:adding 'data_juicer/ops/mapper/generate_qa_from_examples_mapper.py' 2025-03-28T21:37:38,952 INFO:wheel:adding 'data_juicer/ops/mapper/generate_qa_from_text_mapper.py' 2025-03-28T21:37:38,954 INFO:wheel:adding 'data_juicer/ops/mapper/image_blur_mapper.py' 2025-03-28T21:37:38,956 INFO:wheel:adding 'data_juicer/ops/mapper/image_captioning_from_gpt4v_mapper.py' 2025-03-28T21:37:38,958 INFO:wheel:adding 'data_juicer/ops/mapper/image_captioning_mapper.py' 2025-03-28T21:37:38,960 INFO:wheel:adding 'data_juicer/ops/mapper/image_diffusion_mapper.py' 2025-03-28T21:37:38,961 INFO:wheel:adding 'data_juicer/ops/mapper/image_face_blur_mapper.py' 2025-03-28T21:37:38,962 INFO:wheel:adding 'data_juicer/ops/mapper/image_remove_background_mapper.py' 2025-03-28T21:37:38,964 INFO:wheel:adding 'data_juicer/ops/mapper/image_segment_mapper.py' 2025-03-28T21:37:38,965 INFO:wheel:adding 'data_juicer/ops/mapper/image_tagging_mapper.py' 2025-03-28T21:37:38,966 INFO:wheel:adding 'data_juicer/ops/mapper/mllm_mapper.py' 2025-03-28T21:37:38,968 INFO:wheel:adding 'data_juicer/ops/mapper/nlpaug_en_mapper.py' 2025-03-28T21:37:38,970 INFO:wheel:adding 'data_juicer/ops/mapper/nlpcda_zh_mapper.py' 2025-03-28T21:37:38,971 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_qa_mapper.py' 2025-03-28T21:37:38,973 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_query_mapper.py' 2025-03-28T21:37:38,974 INFO:wheel:adding 'data_juicer/ops/mapper/optimize_response_mapper.py' 2025-03-28T21:37:38,975 INFO:wheel:adding 'data_juicer/ops/mapper/pair_preference_mapper.py' 2025-03-28T21:37:38,977 INFO:wheel:adding 'data_juicer/ops/mapper/punctuation_normalization_mapper.py' 2025-03-28T21:37:38,978 INFO:wheel:adding 'data_juicer/ops/mapper/python_file_mapper.py' 2025-03-28T21:37:38,979 INFO:wheel:adding 'data_juicer/ops/mapper/python_lambda_mapper.py' 2025-03-28T21:37:38,980 INFO:wheel:adding 'data_juicer/ops/mapper/query_intent_detection_mapper.py' 2025-03-28T21:37:38,982 INFO:wheel:adding 'data_juicer/ops/mapper/query_sentiment_detection_mapper.py' 2025-03-28T21:37:38,983 INFO:wheel:adding 'data_juicer/ops/mapper/query_topic_detection_mapper.py' 2025-03-28T21:37:38,985 INFO:wheel:adding 'data_juicer/ops/mapper/relation_identity_mapper.py' 2025-03-28T21:37:38,986 INFO:wheel:adding 'data_juicer/ops/mapper/remove_bibliography_mapper.py' 2025-03-28T21:37:38,987 INFO:wheel:adding 'data_juicer/ops/mapper/remove_comments_mapper.py' 2025-03-28T21:37:38,989 INFO:wheel:adding 'data_juicer/ops/mapper/remove_header_mapper.py' 2025-03-28T21:37:38,990 INFO:wheel:adding 'data_juicer/ops/mapper/remove_long_words_mapper.py' 2025-03-28T21:37:38,991 INFO:wheel:adding 'data_juicer/ops/mapper/remove_non_chinese_character_mapper.py' 2025-03-28T21:37:38,993 INFO:wheel:adding 'data_juicer/ops/mapper/remove_repeat_sentences_mapper.py' 2025-03-28T21:37:38,994 INFO:wheel:adding 'data_juicer/ops/mapper/remove_specific_chars_mapper.py' 2025-03-28T21:37:38,995 INFO:wheel:adding 'data_juicer/ops/mapper/remove_table_text_mapper.py' 2025-03-28T21:37:38,997 INFO:wheel:adding 'data_juicer/ops/mapper/remove_words_with_incorrect_substrings_mapper.py' 2025-03-28T21:37:38,999 INFO:wheel:adding 'data_juicer/ops/mapper/replace_content_mapper.py' 2025-03-28T21:37:39,000 INFO:wheel:adding 'data_juicer/ops/mapper/sdxl_prompt2prompt_mapper.py' 2025-03-28T21:37:39,002 INFO:wheel:adding 'data_juicer/ops/mapper/sentence_augmentation_mapper.py' 2025-03-28T21:37:39,004 INFO:wheel:adding 'data_juicer/ops/mapper/sentence_split_mapper.py' 2025-03-28T21:37:39,006 INFO:wheel:adding 'data_juicer/ops/mapper/text_chunk_mapper.py' 2025-03-28T21:37:39,008 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_audio_mapper.py' 2025-03-28T21:37:39,010 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_frames_mapper.py' 2025-03-28T21:37:39,012 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_summarizer_mapper.py' 2025-03-28T21:37:39,015 INFO:wheel:adding 'data_juicer/ops/mapper/video_captioning_from_video_mapper.py' 2025-03-28T21:37:39,016 INFO:wheel:adding 'data_juicer/ops/mapper/video_extract_frames_mapper.py' 2025-03-28T21:37:39,018 INFO:wheel:adding 'data_juicer/ops/mapper/video_face_blur_mapper.py' 2025-03-28T21:37:39,020 INFO:wheel:adding 'data_juicer/ops/mapper/video_ffmpeg_wrapped_mapper.py' 2025-03-28T21:37:39,021 INFO:wheel:adding 'data_juicer/ops/mapper/video_remove_watermark_mapper.py' 2025-03-28T21:37:39,023 INFO:wheel:adding 'data_juicer/ops/mapper/video_resize_aspect_ratio_mapper.py' 2025-03-28T21:37:39,024 INFO:wheel:adding 'data_juicer/ops/mapper/video_resize_resolution_mapper.py' 2025-03-28T21:37:39,026 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_duration_mapper.py' 2025-03-28T21:37:39,027 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_key_frame_mapper.py' 2025-03-28T21:37:39,029 INFO:wheel:adding 'data_juicer/ops/mapper/video_split_by_scene_mapper.py' 2025-03-28T21:37:39,030 INFO:wheel:adding 'data_juicer/ops/mapper/video_tagging_from_audio_mapper.py' 2025-03-28T21:37:39,032 INFO:wheel:adding 'data_juicer/ops/mapper/video_tagging_from_frames_mapper.py' 2025-03-28T21:37:39,033 INFO:wheel:adding 'data_juicer/ops/mapper/whitespace_normalization_mapper.py' 2025-03-28T21:37:39,035 INFO:wheel:adding 'data_juicer/ops/selector/__init__.py' 2025-03-28T21:37:39,036 INFO:wheel:adding 'data_juicer/ops/selector/frequency_specified_field_selector.py' 2025-03-28T21:37:39,037 INFO:wheel:adding 'data_juicer/ops/selector/random_selector.py' 2025-03-28T21:37:39,039 INFO:wheel:adding 'data_juicer/ops/selector/range_specified_field_selector.py' 2025-03-28T21:37:39,040 INFO:wheel:adding 'data_juicer/ops/selector/tags_specified_field_selector.py' 2025-03-28T21:37:39,041 INFO:wheel:adding 'data_juicer/ops/selector/topk_specified_field_selector.py' 2025-03-28T21:37:39,043 INFO:wheel:adding 'data_juicer/tools/__init__.py' 2025-03-28T21:37:39,044 INFO:wheel:adding 'data_juicer/tools/analyze_data.py' 2025-03-28T21:37:39,046 INFO:wheel:adding 'data_juicer/tools/data_resplit.py' 2025-03-28T21:37:39,047 INFO:wheel:adding 'data_juicer/tools/dj_install.py' 2025-03-28T21:37:39,048 INFO:wheel:adding 'data_juicer/tools/process_data.py' 2025-03-28T21:37:39,050 INFO:wheel:adding 'data_juicer/tools/sandbox_starter.py' 2025-03-28T21:37:39,052 INFO:wheel:adding 'data_juicer/utils/__init__.py' 2025-03-28T21:37:39,053 INFO:wheel:adding 'data_juicer/utils/asset_utils.py' 2025-03-28T21:37:39,054 INFO:wheel:adding 'data_juicer/utils/auto_install_mapping.py' 2025-03-28T21:37:39,056 INFO:wheel:adding 'data_juicer/utils/auto_install_utils.py' 2025-03-28T21:37:39,057 INFO:wheel:adding 'data_juicer/utils/availability_utils.py' 2025-03-28T21:37:39,059 INFO:wheel:adding 'data_juicer/utils/cache_utils.py' 2025-03-28T21:37:39,060 INFO:wheel:adding 'data_juicer/utils/ckpt_utils.py' 2025-03-28T21:37:39,062 INFO:wheel:adding 'data_juicer/utils/common_utils.py' 2025-03-28T21:37:39,064 INFO:wheel:adding 'data_juicer/utils/compress.py' 2025-03-28T21:37:39,066 INFO:wheel:adding 'data_juicer/utils/constant.py' 2025-03-28T21:37:39,069 INFO:wheel:adding 'data_juicer/utils/file_utils.py' 2025-03-28T21:37:39,070 INFO:wheel:adding 'data_juicer/utils/fingerprint_utils.py' 2025-03-28T21:37:39,071 INFO:wheel:adding 'data_juicer/utils/lazy_loader.py' 2025-03-28T21:37:39,073 INFO:wheel:adding 'data_juicer/utils/logger_utils.py' 2025-03-28T21:37:39,077 INFO:wheel:adding 'data_juicer/utils/mm_utils.py' 2025-03-28T21:37:39,081 INFO:wheel:adding 'data_juicer/utils/model_utils.py' 2025-03-28T21:37:39,083 INFO:wheel:adding 'data_juicer/utils/process_utils.py' 2025-03-28T21:37:39,084 INFO:wheel:adding 'data_juicer/utils/registry.py' 2025-03-28T21:37:39,086 INFO:wheel:adding 'data_juicer/utils/resource_utils.py' 2025-03-28T21:37:39,087 INFO:wheel:adding 'data_juicer/utils/sample.py' 2025-03-28T21:37:39,088 INFO:wheel:adding 'data_juicer/utils/unittest_utils.py' 2025-03-28T21:37:39,092 INFO:wheel:adding 'py_data_juicer-1.3.0.dist-info/LICENSE' 2025-03-28T21:37:39,097 INFO:wheel:adding 'py_data_juicer-1.3.0.dist-info/METADATA' 2025-03-28T21:37:39,098 INFO:wheel:adding 'py_data_juicer-1.3.0.dist-info/WHEEL' 2025-03-28T21:37:39,099 INFO:wheel:adding 'py_data_juicer-1.3.0.dist-info/entry_points.txt' 2025-03-28T21:37:39,100 INFO:wheel:adding 'py_data_juicer-1.3.0.dist-info/top_level.txt' 2025-03-28T21:37:39,104 INFO:wheel:adding 'py_data_juicer-1.3.0.dist-info/RECORD' 2025-03-28T21:37:39,114 INFO:root:removing build/bdist.linux-armv7l/wheel 2025-03-28T21:37:39,259 Building wheel for py-data-juicer (setup.py): finished with status 'done' 2025-03-28T21:37:39,266 Created wheel for py-data-juicer: filename=py_data_juicer-1.3.0-py3-none-any.whl size=458489 sha256=5ed2d66b66505bb4372f134d7725b8038d7ec3d6ee67df8525608a220c5a8b37 2025-03-28T21:37:39,267 Stored in directory: /tmp/pip-ephem-wheel-cache-7flixiid/wheels/b6/55/42/dabeeaa9fb0307d72adae82964a783a58e0b69820302dac703 2025-03-28T21:37:39,291 Successfully built py-data-juicer 2025-03-28T21:37:39,311 Removed build tracker: '/tmp/pip-build-tracker-q2225lfa'