2022-11-08T12:38:00,659 Created temporary directory: /tmp/pip-ephem-wheel-cache-2ase5rk7 2022-11-08T12:38:00,664 Created temporary directory: /tmp/pip-build-tracker-bbt9fe8y 2022-11-08T12:38:00,664 Initialized build tracking at /tmp/pip-build-tracker-bbt9fe8y 2022-11-08T12:38:00,665 Created build tracker: /tmp/pip-build-tracker-bbt9fe8y 2022-11-08T12:38:00,665 Entered build tracker: /tmp/pip-build-tracker-bbt9fe8y 2022-11-08T12:38:00,666 Created temporary directory: /tmp/pip-wheel-f4rj1jwe 2022-11-08T12:38:00,674 DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453 2022-11-08T12:38:00,679 Created temporary directory: /tmp/pip-ephem-wheel-cache-nleie4mb 2022-11-08T12:38:00,736 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2022-11-08T12:38:00,744 2 location(s) to search for versions of tfds-defect-detection: 2022-11-08T12:38:00,744 * https://pypi.org/simple/tfds-defect-detection/ 2022-11-08T12:38:00,744 * https://www.piwheels.org/simple/tfds-defect-detection/ 2022-11-08T12:38:00,745 Fetching project page and analyzing links: https://pypi.org/simple/tfds-defect-detection/ 2022-11-08T12:38:00,746 Getting page https://pypi.org/simple/tfds-defect-detection/ 2022-11-08T12:38:00,749 Found index url https://pypi.org/simple 2022-11-08T12:38:00,971 Fetched page https://pypi.org/simple/tfds-defect-detection/ as application/vnd.pypi.simple.v1+json 2022-11-08T12:38:00,974 Found link https://files.pythonhosted.org/packages/c7/3d/de175feaae9a336e8579223f293690c37a13510faf8b0e5fec165cd1eef9/tfds_defect_detection-0.1.0.tar.gz (from https://pypi.org/simple/tfds-defect-detection/), version: 0.1.0 2022-11-08T12:38:00,975 Found link https://files.pythonhosted.org/packages/70/b3/ef458615ff67df146bedbc07f891920727a545cd4759e3d82eabd78c540c/tfds_defect_detection-1.0.0.tar.gz (from https://pypi.org/simple/tfds-defect-detection/), version: 1.0.0 2022-11-08T12:38:00,976 Fetching project page and analyzing links: https://www.piwheels.org/simple/tfds-defect-detection/ 2022-11-08T12:38:00,976 Getting page https://www.piwheels.org/simple/tfds-defect-detection/ 2022-11-08T12:38:00,978 Found index url https://www.piwheels.org/simple 2022-11-08T12:38:01,199 Fetched page https://www.piwheels.org/simple/tfds-defect-detection/ as text/html 2022-11-08T12:38:01,202 Skipping link: No binaries permitted for tfds-defect-detection: https://www.piwheels.org/simple/tfds-defect-detection/tfds_defect_detection-0.1.0-py3-none-any.whl#sha256=b68e65517f554748d712322c547452ca0b609b1d80f6f04bc85b6f21d0ae590d (from https://www.piwheels.org/simple/tfds-defect-detection/) 2022-11-08T12:38:01,202 Skipping link: not a file: https://www.piwheels.org/simple/tfds-defect-detection/ 2022-11-08T12:38:01,203 Skipping link: not a file: https://pypi.org/simple/tfds-defect-detection/ 2022-11-08T12:38:01,236 Given no hashes to check 1 links for project 'tfds-defect-detection': discarding no candidates 2022-11-08T12:38:01,270 Collecting tfds-defect-detection==1.0.0 2022-11-08T12:38:01,274 Created temporary directory: /tmp/pip-unpack-bf_yiz6z 2022-11-08T12:38:01,507 Downloading tfds_defect_detection-1.0.0.tar.gz (19 kB) 2022-11-08T12:38:01,596 Added tfds-defect-detection==1.0.0 from https://files.pythonhosted.org/packages/70/b3/ef458615ff67df146bedbc07f891920727a545cd4759e3d82eabd78c540c/tfds_defect_detection-1.0.0.tar.gz to build tracker '/tmp/pip-build-tracker-bbt9fe8y' 2022-11-08T12:38:01,599 Running setup.py (path:/tmp/pip-wheel-f4rj1jwe/tfds-defect-detection_200a542bf5704cc2b42af189ceb789a3/setup.py) egg_info for package tfds-defect-detection 2022-11-08T12:38:01,600 Created temporary directory: /tmp/pip-pip-egg-info-k6w_xz8h 2022-11-08T12:38:01,601 Preparing metadata (setup.py): started 2022-11-08T12:38:01,603 Running command python setup.py egg_info 2022-11-08T12:38:04,033 .. figure:: tfds_defect_detection/assets/images/logo.png 2022-11-08T12:38:04,033 :align: center 2022-11-08T12:38:04,034 :alt: 2022-11-08T12:38:04,034 :scale: 50 % 2022-11-08T12:38:04,035 .. image:: https://readthedocs.org/projects/tfds-defect-detection/badge/?version=latest 2022-11-08T12:38:04,035 :target: https://tfds-defect-detection.readthedocs.io/en/latest/README.html 2022-11-08T12:38:04,036 :alt: Documentation Status 2022-11-08T12:38:04,036 .. image:: https://img.shields.io/pypi/v/tfds_defect_detection 2022-11-08T12:38:04,037 :target: https://pypi.org/project/tfds-defect-detection/ 2022-11-08T12:38:04,037 .. image:: https://img.shields.io/pypi/pyversions/tfds_defect_detection 2022-11-08T12:38:04,038 :alt: PyPI - Python Version 2022-11-08T12:38:04,039 ======================================== 2022-11-08T12:38:04,039 TensorFlow Datasets for Defect Detection 2022-11-08T12:38:04,040 ======================================== 2022-11-08T12:38:04,040 To directly jump into the code look at the sample notebook 2022-11-08T12:38:04,041 .. class:: center 2022-11-08T12:38:04,042 |Open in Colab| 2022-11-08T12:38:04,043 .. |Open in Colab| image:: https://img.shields.io/badge/Open%20In-Colab-orange?style=for-the-badge&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAB4AAAAUCAYAAACaq43EAAAACXBIWXMAAAdiAAAHYgE4epnbAAAEg2lUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPD94cGFja2V0IGJlZ2luPSfvu78nIGlkPSdXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQnPz4KPHg6eG1wbWV0YSB4bWxuczp4PSdhZG9iZTpuczptZXRhLyc+CjxyZGY6UkRGIHhtbG5zOnJkZj0naHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyc+CgogPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9JycKICB4bWxuczpBdHRyaWI9J2h0dHA6Ly9ucy5hdHRyaWJ1dGlvbi5jb20vYWRzLzEuMC8nPgogIDxBdHRyaWI6QWRzPgogICA8cmRmOlNlcT4KICAgIDxyZGY6bGkgcmRmOnBhcnNlVHlwZT0nUmVzb3VyY2UnPgogICAgIDxBdHRyaWI6Q3JlYXRlZD4yMDIyLTExLTA4PC9BdHRyaWI6Q3JlYXRlZD4KICAgICA8QXR0cmliOkV4dElkPjdiZTBlMWY0LTU3YTUtNDBhMi05ZjIzLTM2N2Q1YzU1OGYyNjwvQXR0cmliOkV4dElkPgogICAgIDxBdHRyaWI6RmJJZD41MjUyNjU5MTQxNzk1ODA8L0F0dHJpYjpGYklkPgogICAgIDxBdHRyaWI6VG91Y2hUeXBlPjI8L0F0dHJpYjpUb3VjaFR5cGU+CiAgICA8L3JkZjpsaT4KICAgPC9yZGY6U2VxPgogIDwvQXR0cmliOkFkcz4KIDwvcmRmOkRlc2NyaXB0aW9uPgoKIDxyZGY6RGVzY3JpcHRpb24gcmRmOmFib3V0PScnCiAgeG1sbnM6ZGM9J2h0dHA6Ly9wdXJsLm9yZy9kYy9lbGVtZW50cy8xLjEvJz4KICA8ZGM6dGl0bGU+CiAgIDxyZGY6QWx0PgogICAgPHJkZjpsaSB4bWw6bGFuZz0neC1kZWZhdWx0Jz5PUEVOIElOIENPTEFCICgxMDAgw5cgMTAwIHB4KSAtIDE8L3JkZjpsaT4KICAgPC9yZGY6QWx0PgogIDwvZGM6dGl0bGU+CiA8L3JkZjpEZXNjcmlwdGlvbj4KCiA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0nJwogIHhtbG5zOnBkZj0naHR0cDovL25zLmFkb2JlLmNvbS9wZGYvMS4zLyc+CiAgPHBkZjpBdXRob3I+VG9iaWFzIFNjaGllbGU8L3BkZjpBdXRob3I+CiA8L3JkZjpEZXNjcmlwdGlvbj4KCiA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0nJwogIHhtbG5zOnhtcD0naHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLyc+CiAgPHhtcDpDcmVhdG9yVG9vbD5DYW52YTwveG1wOkNyZWF0b3JUb29sPgogPC9yZGY6RGVzY3JpcHRpb24+CjwvcmRmOlJERj4KPC94OnhtcG1ldGE+Cjw/eHBhY2tldCBlbmQ9J3InPz5sF+fkAAAFSElEQVRIiV2Va4heRx3Gf8/MOe9599I2u61dNyaCTc02NZUi1iimrRb9oFCptShVqYgXFIsaEEP9UIt+EUHwgmCiVjGIFupKEBsbQWKlWKL9oCRNixhtYtKtYbPp3vK+7zkzjx/O2aZmDsMwM//L/C/Pc/TXH8BN0xAAIqREiBE3DVYAzKWRgUB7KkwG1H2ZjPj/YQgFygmtrJOvnoAa6BmKG4pAumhG66KczgHIna8tmDcDrwIScAo4CizbEAIYIiYhjJlG7AK2AALOAUdtzgCe7BNWX4w5TmRGIxGa5UBal9742QeIY2TE23LioMQJ4NfAfuDHwO8JPAd8B3jtaBkQSeJ1mB92d4918vuAecQJZ+aBW+I4+dkvz5CGIq0FtPy9QLk5s/DzK9n8keWHnPiqLqUsdVNtIQhu0/FPNnE9F7gH+Akw2elkoOl0YzexQQUPnDs08Y3R0T7XfGaJuPc9hQb/LolXpG8Wk/krxQR2ppEwUNhEQUQIGAAF8C0GTAHzEj2JUdcN0W4dSgSgsUmhJNaL4V3NSlC1fXQEK2jhi30MH8R6RIXrK3YOc3GVK7fvXgD+AVTAjcAk8AzwceAPEhM2jUS0EbAMHOuingNmVEC9pPXVY/3SmRL5fcBvgoZNBfqaArgJWjleVfWSahXsTeuaqxe47dl1dtnsBA4CDwGfAiaA+hVO92N2hB5vr3Zwe1rXDSp4sF4Mo9XjVc9ZUgDQ1z1IUS98oX830q9ASbKcCeD7qtl0YOpjNTwPC6dhUwsllLkOcUxirEtlgfk2kT0kUIDekR58fsTSfMnwhfhppH0KZCwbR+z3BsRtABKNTUAcDkU4UE07rB2SRqdgyiBRqm2sW6F12tX7DOZBN0AbPcN3jlh7QmFiLkll3I/8x852W0Dp1gB6w2Ww/12qzcpzZehtxvWqUGu0TajY3sk1Hd6PqGAlRIJNAqgXA3Gb89KTvZibDNbhl6233X9TAMa7TeiuVhSMEz5/aJLVv/c2VDY4LL68b0/W7BYyQeARrJ8oOf1wgZNoYeb1y4IbC8DpzkBqzfnGtB4pp1KY2bvCNXcMsdkgSTAvdcobD91ebYGlp/qECv780WsJWVw1N3YpPrHtsseeCdhPbxiSBNKHB2d0bRjL9dpjoWwGBMqWPDq5P3VrqZYq31H/l9vHXtPk5WNlb8ee5VgHQl5seti1nGdB93ahd9ny0UBLczXQAxKZmendwwNTn2wmq1fnmpaNEjDquPscLZaDTVIJo3Nh/8rx3s7BmXJEVgKy8Mj2Jku/AK5W24wlcBH7cPzS7vIcZpvEzblW059t6vHX13PpJB/yiB5iCtgJfALxU4QwjwJ3qSDXS7q4eqKaDSX3qfAUYkLmeqT3S/oR6GapfYztCH6YEH6ms/f3CaVnnMNfqplm68RcPSS3VHlZQyC1TWSzO0T21Bf0gdVnqjWnMKbgDR6/JN8qNZ3THnAS51tA54tqNkU3ehE1d45fVz9OZsYmSwy59HNwl24DfYnv9qa5Y+mpasZJuxXA1miD3zu/qVsL2wXwH+w7kc4DIYzOxxT6jqHvv6ngLc4c7OpX2fQ6xwXtvg+sYh5f/G310uBkfDfy920ncA9cYUfsaLvXzQCed867ut6ISLnITSCO1ykPFBU55cxdtOx0N/Amm5mu+Z7HPAn80vCvxeN9gjzoh3x/Tt4nca+ttwJbOwQsgJ/GPErQE13eQ5sJocEjrywIQcLu0FaXUAzpYRwK6g7PWIT+PeSznxtTHEM5OUtCV06QLqxUAJwdjuLWMdsGBZEzpiVdkfkfodOPwQgKJiAAAAAASUVORK5CYII= 2022-11-08T12:38:04,043 :target: https://colab.research.google.com/drive/1_0diKQAHBX2q8iCEI7bmv0TnnmaWZR1M?usp=sharing 2022-11-08T12:38:04,044 .. admonition:: Features 2022-11-08T12:38:04,045 - tensorflow.data.Dataset builder for defect segmentation 2022-11-08T12:38:04,045 - Comes with unsupervised / self-supervised SotA datasets 2022-11-08T12:38:04,045 - MVTEC 2022-11-08T12:38:04,046 - VISA 2022-11-08T12:38:04,046 - Artificial defect generator 2022-11-08T12:38:04,046 - Evaluation data with hand labelled images 2022-11-08T12:38:04,047 Install 2022-11-08T12:38:04,048 ------- 2022-11-08T12:38:04,048 Create a new python=3.9 env and install `tfds_defect_detection` from pip 2022-11-08T12:38:04,049 .. code-block:: bash 2022-11-08T12:38:04,050 pip install tfds_defect_detection 2022-11-08T12:38:04,052 Examples 2022-11-08T12:38:04,052 ----------- 2022-11-08T12:38:04,053 .. code-block:: python 2022-11-08T12:38:04,053 import tfds_defect_detection as tfd 2022-11-08T12:38:04,054 tfd.load() 2022-11-08T12:38:04,055 Usage 2022-11-08T12:38:04,055 ----------- 2022-11-08T12:38:04,056 All parmeters 2022-11-08T12:38:04,057 .. code-block:: python 2022-11-08T12:38:04,058 import tfds_defect_detection as tfd 2022-11-08T12:38:04,058 impor albumentations as A 2022-11-08T12:38:04,059 ds = tfd.load( 2022-11-08T12:38:04,059 names = ("mvtec", "visa"), 2022-11-08T12:38:04,059 data_dir=Path("."), 2022-11-08T12:38:04,060 pairing_mode = "result_with_contrastive_pair", # "result_only", "result_with_original" 2022-11-08T12:38:04,060 create_artificial_anomalies=True, 2022-11-08T12:38:04,060 validation_split=0.2, 2022-11-08T12:38:04,061 subset_mode = "training", # "validation", "test", "holdout", None 2022-11-08T12:38:04,061 drop_masks=False, 2022-11-08T12:38:04,062 width=256, 2022-11-08T12:38:04,062 height=256, 2022-11-08T12:38:04,062 repeat=True, 2022-11-08T12:38:04,063 anomaly_size = None, 2022-11-08T12:38:04,063 global_transform=A.Compose([ 2022-11-08T12:38:04,063 A.RandomBrightnessContrast(), 2022-11-08T12:38:04,064 A.HueSaturationValue(), 2022-11-08T12:38:04,064 ]), 2022-11-08T12:38:04,065 process_deviation=A.Compose([ 2022-11-08T12:38:04,065 A.ShiftScaleRotate( 2022-11-08T12:38:04,066 shift_limit=0.01, 2022-11-08T12:38:04,066 scale_limit=0.0, 2022-11-08T12:38:04,066 rotate_limit=1.5, 2022-11-08T12:38:04,067 p=1 2022-11-08T12:38:04,067 ), 2022-11-08T12:38:04,067 A.Blur(blur_limit=3), 2022-11-08T12:38:04,068 A.RandomBrightnessContrast(), 2022-11-08T12:38:04,068 A.RandomGamma(), 2022-11-08T12:38:04,068 A.HueSaturationValue(), 2022-11-08T12:38:04,068 ]), 2022-11-08T12:38:04,069 anomaly_composition=A.Compose([ 2022-11-08T12:38:04,069 A.RandomRotate90(), 2022-11-08T12:38:04,070 A.Transpose(), 2022-11-08T12:38:04,070 A.ShiftScaleRotate( 2022-11-08T12:38:04,070 shift_limit=0.0625, 2022-11-08T12:38:04,071 scale_limit=0.50, 2022-11-08T12:38:04,071 rotate_limit=45, p=1 2022-11-08T12:38:04,071 ), 2022-11-08T12:38:04,072 A.RandomGamma(), 2022-11-08T12:38:04,072 A.OpticalDistortion(), 2022-11-08T12:38:04,073 A.GridDistortion(), 2022-11-08T12:38:04,073 A.RandomContrast(0.5, p=1), 2022-11-08T12:38:04,073 ]), 2022-11-08T12:38:04,074 batch_size=9, 2022-11-08T12:38:04,074 seed=123, 2022-11-08T12:38:04,075 shuffle=True, 2022-11-08T12:38:04,075 peek=True, 2022-11-08T12:38:04,075 image_validation=False, 2022-11-08T12:38:04,076 delete_tmp=True, 2022-11-08T12:38:04,076 crop_to_aspect_ratio=True 2022-11-08T12:38:04,076 ) 2022-11-08T12:38:04,077 .. figure:: tfds_defect_detection/assets/images/example.png 2022-11-08T12:38:04,077 :align: center 2022-11-08T12:38:04,078 :alt: 2022-11-08T12:38:04,078 :scale: 50 % 2022-11-08T12:38:04,079 .. admonition:: Docs 2022-11-08T12:38:04,080 FOR API Reference see 2022-11-08T12:38:04,081 https://tfds-defect-detection.readthedocs.io/en/latest/autoapi/tfds_defect_detection/index.html 2022-11-08T12:38:04,082 .. admonition:: Cite 2022-11-08T12:38:04,083 If this project helped you during your work: 2022-11-08T12:38:04,083 Until a publication is available, please cite as 2022-11-08T12:38:04,084 Tobias Schiele. (2022). TFDS DD - Datasets for Defect Detection. https://github.com/thetoby9944/tfds_defect_detection. 2022-11-08T12:38:04,085 .. code-block:: latex 2022-11-08T12:38:04,086 @misc{Schiele2019, 2022-11-08T12:38:04,086 author = {Tobias Schiele}, 2022-11-08T12:38:04,086 title = {TFDS DD - Datasets for Defect Detection}, 2022-11-08T12:38:04,087 year = {2022}, 2022-11-08T12:38:04,087 publisher = {GitHub}, 2022-11-08T12:38:04,087 journal = {GitHub repository}, 2022-11-08T12:38:04,088 howpublished = {\url{https://github.com/thetoby9944/tfds_defect_detection}}, 2022-11-08T12:38:04,088 } 2022-11-08T12:38:04,089 running egg_info 2022-11-08T12:38:04,090 creating /tmp/pip-pip-egg-info-k6w_xz8h/tfds_defect_detection.egg-info 2022-11-08T12:38:04,103 writing /tmp/pip-pip-egg-info-k6w_xz8h/tfds_defect_detection.egg-info/PKG-INFO 2022-11-08T12:38:04,108 writing dependency_links to /tmp/pip-pip-egg-info-k6w_xz8h/tfds_defect_detection.egg-info/dependency_links.txt 2022-11-08T12:38:04,112 writing requirements to /tmp/pip-pip-egg-info-k6w_xz8h/tfds_defect_detection.egg-info/requires.txt 2022-11-08T12:38:04,114 writing top-level names to /tmp/pip-pip-egg-info-k6w_xz8h/tfds_defect_detection.egg-info/top_level.txt 2022-11-08T12:38:04,116 writing manifest file '/tmp/pip-pip-egg-info-k6w_xz8h/tfds_defect_detection.egg-info/SOURCES.txt' 2022-11-08T12:38:04,307 reading manifest file '/tmp/pip-pip-egg-info-k6w_xz8h/tfds_defect_detection.egg-info/SOURCES.txt' 2022-11-08T12:38:04,310 reading manifest template 'MANIFEST.in' 2022-11-08T12:38:04,316 writing manifest file '/tmp/pip-pip-egg-info-k6w_xz8h/tfds_defect_detection.egg-info/SOURCES.txt' 2022-11-08T12:38:04,473 Preparing metadata (setup.py): finished with status 'done' 2022-11-08T12:38:04,487 Source in /tmp/pip-wheel-f4rj1jwe/tfds-defect-detection_200a542bf5704cc2b42af189ceb789a3 has version 1.0.0, which satisfies requirement tfds-defect-detection==1.0.0 from https://files.pythonhosted.org/packages/70/b3/ef458615ff67df146bedbc07f891920727a545cd4759e3d82eabd78c540c/tfds_defect_detection-1.0.0.tar.gz 2022-11-08T12:38:04,489 Removed tfds-defect-detection==1.0.0 from https://files.pythonhosted.org/packages/70/b3/ef458615ff67df146bedbc07f891920727a545cd4759e3d82eabd78c540c/tfds_defect_detection-1.0.0.tar.gz from build tracker '/tmp/pip-build-tracker-bbt9fe8y' 2022-11-08T12:38:04,501 Created temporary directory: /tmp/pip-unpack-7m8g3vzr 2022-11-08T12:38:04,502 Building wheels for collected packages: tfds-defect-detection 2022-11-08T12:38:04,512 Created temporary directory: /tmp/pip-wheel-5vb7y_ta 2022-11-08T12:38:04,513 Building wheel for tfds-defect-detection (setup.py): started 2022-11-08T12:38:04,515 Destination directory: /tmp/pip-wheel-5vb7y_ta 2022-11-08T12:38:04,515 Running command python setup.py bdist_wheel 2022-11-08T12:38:06,501 .. figure:: tfds_defect_detection/assets/images/logo.png 2022-11-08T12:38:06,502 :align: center 2022-11-08T12:38:06,502 :alt: 2022-11-08T12:38:06,503 :scale: 50 % 2022-11-08T12:38:06,504 .. image:: https://readthedocs.org/projects/tfds-defect-detection/badge/?version=latest 2022-11-08T12:38:06,504 :target: https://tfds-defect-detection.readthedocs.io/en/latest/README.html 2022-11-08T12:38:06,504 :alt: Documentation Status 2022-11-08T12:38:06,505 .. image:: https://img.shields.io/pypi/v/tfds_defect_detection 2022-11-08T12:38:06,505 :target: https://pypi.org/project/tfds-defect-detection/ 2022-11-08T12:38:06,506 .. image:: https://img.shields.io/pypi/pyversions/tfds_defect_detection 2022-11-08T12:38:06,506 :alt: PyPI - Python Version 2022-11-08T12:38:06,507 ======================================== 2022-11-08T12:38:06,507 TensorFlow Datasets for Defect Detection 2022-11-08T12:38:06,507 ======================================== 2022-11-08T12:38:06,508 To directly jump into the code look at the sample notebook 2022-11-08T12:38:06,509 .. class:: center 2022-11-08T12:38:06,509 |Open in Colab| 2022-11-08T12:38:06,510 .. |Open in Colab| image:: https://img.shields.io/badge/Open%20In-Colab-orange?style=for-the-badge&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAB4AAAAUCAYAAACaq43EAAAACXBIWXMAAAdiAAAHYgE4epnbAAAEg2lUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPD94cGFja2V0IGJlZ2luPSfvu78nIGlkPSdXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQnPz4KPHg6eG1wbWV0YSB4bWxuczp4PSdhZG9iZTpuczptZXRhLyc+CjxyZGY6UkRGIHhtbG5zOnJkZj0naHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyc+CgogPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9JycKICB4bWxuczpBdHRyaWI9J2h0dHA6Ly9ucy5hdHRyaWJ1dGlvbi5jb20vYWRzLzEuMC8nPgogIDxBdHRyaWI6QWRzPgogICA8cmRmOlNlcT4KICAgIDxyZGY6bGkgcmRmOnBhcnNlVHlwZT0nUmVzb3VyY2UnPgogICAgIDxBdHRyaWI6Q3JlYXRlZD4yMDIyLTExLTA4PC9BdHRyaWI6Q3JlYXRlZD4KICAgICA8QXR0cmliOkV4dElkPjdiZTBlMWY0LTU3YTUtNDBhMi05ZjIzLTM2N2Q1YzU1OGYyNjwvQXR0cmliOkV4dElkPgogICAgIDxBdHRyaWI6RmJJZD41MjUyNjU5MTQxNzk1ODA8L0F0dHJpYjpGYklkPgogICAgIDxBdHRyaWI6VG91Y2hUeXBlPjI8L0F0dHJpYjpUb3VjaFR5cGU+CiAgICA8L3JkZjpsaT4KICAgPC9yZGY6U2VxPgogIDwvQXR0cmliOkFkcz4KIDwvcmRmOkRlc2NyaXB0aW9uPgoKIDxyZGY6RGVzY3JpcHRpb24gcmRmOmFib3V0PScnCiAgeG1sbnM6ZGM9J2h0dHA6Ly9wdXJsLm9yZy9kYy9lbGVtZW50cy8xLjEvJz4KICA8ZGM6dGl0bGU+CiAgIDxyZGY6QWx0PgogICAgPHJkZjpsaSB4bWw6bGFuZz0neC1kZWZhdWx0Jz5PUEVOIElOIENPTEFCICgxMDAgw5cgMTAwIHB4KSAtIDE8L3JkZjpsaT4KICAgPC9yZGY6QWx0PgogIDwvZGM6dGl0bGU+CiA8L3JkZjpEZXNjcmlwdGlvbj4KCiA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0nJwogIHhtbG5zOnBkZj0naHR0cDovL25zLmFkb2JlLmNvbS9wZGYvMS4zLyc+CiAgPHBkZjpBdXRob3I+VG9iaWFzIFNjaGllbGU8L3BkZjpBdXRob3I+CiA8L3JkZjpEZXNjcmlwdGlvbj4KCiA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0nJwogIHhtbG5zOnhtcD0naHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLyc+CiAgPHhtcDpDcmVhdG9yVG9vbD5DYW52YTwveG1wOkNyZWF0b3JUb29sPgogPC9yZGY6RGVzY3JpcHRpb24+CjwvcmRmOlJERj4KPC94OnhtcG1ldGE+Cjw/eHBhY2tldCBlbmQ9J3InPz5sF+fkAAAFSElEQVRIiV2Va4heRx3Gf8/MOe9599I2u61dNyaCTc02NZUi1iimrRb9oFCptShVqYgXFIsaEEP9UIt+EUHwgmCiVjGIFupKEBsbQWKlWKL9oCRNixhtYtKtYbPp3vK+7zkzjx/O2aZmDsMwM//L/C/Pc/TXH8BN0xAAIqREiBE3DVYAzKWRgUB7KkwG1H2ZjPj/YQgFygmtrJOvnoAa6BmKG4pAumhG66KczgHIna8tmDcDrwIScAo4CizbEAIYIiYhjJlG7AK2AALOAUdtzgCe7BNWX4w5TmRGIxGa5UBal9742QeIY2TE23LioMQJ4NfAfuDHwO8JPAd8B3jtaBkQSeJ1mB92d4918vuAecQJZ+aBW+I4+dkvz5CGIq0FtPy9QLk5s/DzK9n8keWHnPiqLqUsdVNtIQhu0/FPNnE9F7gH+Akw2elkoOl0YzexQQUPnDs08Y3R0T7XfGaJuPc9hQb/LolXpG8Wk/krxQR2ppEwUNhEQUQIGAAF8C0GTAHzEj2JUdcN0W4dSgSgsUmhJNaL4V3NSlC1fXQEK2jhi30MH8R6RIXrK3YOc3GVK7fvXgD+AVTAjcAk8AzwceAPEhM2jUS0EbAMHOuingNmVEC9pPXVY/3SmRL5fcBvgoZNBfqaArgJWjleVfWSahXsTeuaqxe47dl1dtnsBA4CDwGfAiaA+hVO92N2hB5vr3Zwe1rXDSp4sF4Mo9XjVc9ZUgDQ1z1IUS98oX830q9ASbKcCeD7qtl0YOpjNTwPC6dhUwsllLkOcUxirEtlgfk2kT0kUIDekR58fsTSfMnwhfhppH0KZCwbR+z3BsRtABKNTUAcDkU4UE07rB2SRqdgyiBRqm2sW6F12tX7DOZBN0AbPcN3jlh7QmFiLkll3I/8x852W0Dp1gB6w2Ww/12qzcpzZehtxvWqUGu0TajY3sk1Hd6PqGAlRIJNAqgXA3Gb89KTvZibDNbhl6233X9TAMa7TeiuVhSMEz5/aJLVv/c2VDY4LL68b0/W7BYyQeARrJ8oOf1wgZNoYeb1y4IbC8DpzkBqzfnGtB4pp1KY2bvCNXcMsdkgSTAvdcobD91ebYGlp/qECv780WsJWVw1N3YpPrHtsseeCdhPbxiSBNKHB2d0bRjL9dpjoWwGBMqWPDq5P3VrqZYq31H/l9vHXtPk5WNlb8ee5VgHQl5seti1nGdB93ahd9ny0UBLczXQAxKZmendwwNTn2wmq1fnmpaNEjDquPscLZaDTVIJo3Nh/8rx3s7BmXJEVgKy8Mj2Jku/AK5W24wlcBH7cPzS7vIcZpvEzblW059t6vHX13PpJB/yiB5iCtgJfALxU4QwjwJ3qSDXS7q4eqKaDSX3qfAUYkLmeqT3S/oR6GapfYztCH6YEH6ms/f3CaVnnMNfqplm68RcPSS3VHlZQyC1TWSzO0T21Bf0gdVnqjWnMKbgDR6/JN8qNZ3THnAS51tA54tqNkU3ehE1d45fVz9OZsYmSwy59HNwl24DfYnv9qa5Y+mpasZJuxXA1miD3zu/qVsL2wXwH+w7kc4DIYzOxxT6jqHvv6ngLc4c7OpX2fQ6xwXtvg+sYh5f/G310uBkfDfy920ncA9cYUfsaLvXzQCed867ut6ISLnITSCO1ykPFBU55cxdtOx0N/Amm5mu+Z7HPAn80vCvxeN9gjzoh3x/Tt4nca+ttwJbOwQsgJ/GPErQE13eQ5sJocEjrywIQcLu0FaXUAzpYRwK6g7PWIT+PeSznxtTHEM5OUtCV06QLqxUAJwdjuLWMdsGBZEzpiVdkfkfodOPwQgKJiAAAAAASUVORK5CYII= 2022-11-08T12:38:06,511 :target: https://colab.research.google.com/drive/1_0diKQAHBX2q8iCEI7bmv0TnnmaWZR1M?usp=sharing 2022-11-08T12:38:06,511 .. admonition:: Features 2022-11-08T12:38:06,512 - tensorflow.data.Dataset builder for defect segmentation 2022-11-08T12:38:06,513 - Comes with unsupervised / self-supervised SotA datasets 2022-11-08T12:38:06,513 - MVTEC 2022-11-08T12:38:06,513 - VISA 2022-11-08T12:38:06,514 - Artificial defect generator 2022-11-08T12:38:06,514 - Evaluation data with hand labelled images 2022-11-08T12:38:06,515 Install 2022-11-08T12:38:06,515 ------- 2022-11-08T12:38:06,516 Create a new python=3.9 env and install `tfds_defect_detection` from pip 2022-11-08T12:38:06,516 .. code-block:: bash 2022-11-08T12:38:06,517 pip install tfds_defect_detection 2022-11-08T12:38:06,519 Examples 2022-11-08T12:38:06,519 ----------- 2022-11-08T12:38:06,520 .. code-block:: python 2022-11-08T12:38:06,520 import tfds_defect_detection as tfd 2022-11-08T12:38:06,521 tfd.load() 2022-11-08T12:38:06,522 Usage 2022-11-08T12:38:06,523 ----------- 2022-11-08T12:38:06,523 All parmeters 2022-11-08T12:38:06,524 .. code-block:: python 2022-11-08T12:38:06,525 import tfds_defect_detection as tfd 2022-11-08T12:38:06,525 impor albumentations as A 2022-11-08T12:38:06,526 ds = tfd.load( 2022-11-08T12:38:06,526 names = ("mvtec", "visa"), 2022-11-08T12:38:06,527 data_dir=Path("."), 2022-11-08T12:38:06,527 pairing_mode = "result_with_contrastive_pair", # "result_only", "result_with_original" 2022-11-08T12:38:06,528 create_artificial_anomalies=True, 2022-11-08T12:38:06,528 validation_split=0.2, 2022-11-08T12:38:06,528 subset_mode = "training", # "validation", "test", "holdout", None 2022-11-08T12:38:06,529 drop_masks=False, 2022-11-08T12:38:06,529 width=256, 2022-11-08T12:38:06,529 height=256, 2022-11-08T12:38:06,529 repeat=True, 2022-11-08T12:38:06,530 anomaly_size = None, 2022-11-08T12:38:06,530 global_transform=A.Compose([ 2022-11-08T12:38:06,531 A.RandomBrightnessContrast(), 2022-11-08T12:38:06,531 A.HueSaturationValue(), 2022-11-08T12:38:06,531 ]), 2022-11-08T12:38:06,532 process_deviation=A.Compose([ 2022-11-08T12:38:06,533 A.ShiftScaleRotate( 2022-11-08T12:38:06,533 shift_limit=0.01, 2022-11-08T12:38:06,533 scale_limit=0.0, 2022-11-08T12:38:06,534 rotate_limit=1.5, 2022-11-08T12:38:06,534 p=1 2022-11-08T12:38:06,534 ), 2022-11-08T12:38:06,534 A.Blur(blur_limit=3), 2022-11-08T12:38:06,535 A.RandomBrightnessContrast(), 2022-11-08T12:38:06,535 A.RandomGamma(), 2022-11-08T12:38:06,535 A.HueSaturationValue(), 2022-11-08T12:38:06,536 ]), 2022-11-08T12:38:06,536 anomaly_composition=A.Compose([ 2022-11-08T12:38:06,536 A.RandomRotate90(), 2022-11-08T12:38:06,537 A.Transpose(), 2022-11-08T12:38:06,537 A.ShiftScaleRotate( 2022-11-08T12:38:06,537 shift_limit=0.0625, 2022-11-08T12:38:06,538 scale_limit=0.50, 2022-11-08T12:38:06,538 rotate_limit=45, p=1 2022-11-08T12:38:06,538 ), 2022-11-08T12:38:06,538 A.RandomGamma(), 2022-11-08T12:38:06,539 A.OpticalDistortion(), 2022-11-08T12:38:06,539 A.GridDistortion(), 2022-11-08T12:38:06,539 A.RandomContrast(0.5, p=1), 2022-11-08T12:38:06,540 ]), 2022-11-08T12:38:06,540 batch_size=9, 2022-11-08T12:38:06,540 seed=123, 2022-11-08T12:38:06,541 shuffle=True, 2022-11-08T12:38:06,541 peek=True, 2022-11-08T12:38:06,542 image_validation=False, 2022-11-08T12:38:06,542 delete_tmp=True, 2022-11-08T12:38:06,542 crop_to_aspect_ratio=True 2022-11-08T12:38:06,542 ) 2022-11-08T12:38:06,544 .. figure:: tfds_defect_detection/assets/images/example.png 2022-11-08T12:38:06,544 :align: center 2022-11-08T12:38:06,544 :alt: 2022-11-08T12:38:06,545 :scale: 50 % 2022-11-08T12:38:06,546 .. admonition:: Docs 2022-11-08T12:38:06,547 FOR API Reference see 2022-11-08T12:38:06,547 https://tfds-defect-detection.readthedocs.io/en/latest/autoapi/tfds_defect_detection/index.html 2022-11-08T12:38:06,548 .. admonition:: Cite 2022-11-08T12:38:06,549 If this project helped you during your work: 2022-11-08T12:38:06,549 Until a publication is available, please cite as 2022-11-08T12:38:06,550 Tobias Schiele. (2022). TFDS DD - Datasets for Defect Detection. https://github.com/thetoby9944/tfds_defect_detection. 2022-11-08T12:38:06,551 .. code-block:: latex 2022-11-08T12:38:06,551 @misc{Schiele2019, 2022-11-08T12:38:06,552 author = {Tobias Schiele}, 2022-11-08T12:38:06,552 title = {TFDS DD - Datasets for Defect Detection}, 2022-11-08T12:38:06,552 year = {2022}, 2022-11-08T12:38:06,553 publisher = {GitHub}, 2022-11-08T12:38:06,553 journal = {GitHub repository}, 2022-11-08T12:38:06,553 howpublished = {\url{https://github.com/thetoby9944/tfds_defect_detection}}, 2022-11-08T12:38:06,554 } 2022-11-08T12:38:06,933 running bdist_wheel 2022-11-08T12:38:07,681 running build 2022-11-08T12:38:07,682 running build_py 2022-11-08T12:38:07,749 creating build 2022-11-08T12:38:07,750 creating build/lib 2022-11-08T12:38:07,751 creating build/lib/tfds_defect_detection 2022-11-08T12:38:07,753 copying tfds_defect_detection/__init__.py -> build/lib/tfds_defect_detection 2022-11-08T12:38:07,758 copying tfds_defect_detection/test.py -> build/lib/tfds_defect_detection 2022-11-08T12:38:07,762 copying tfds_defect_detection/utils.py -> build/lib/tfds_defect_detection 2022-11-08T12:38:07,766 copying tfds_defect_detection/data.py -> build/lib/tfds_defect_detection 2022-11-08T12:38:07,772 creating build/lib/tfds_defect_detection/downloader 2022-11-08T12:38:07,774 copying tfds_defect_detection/downloader/__init__.py -> build/lib/tfds_defect_detection/downloader 2022-11-08T12:38:07,778 copying tfds_defect_detection/downloader/visual_anomalies.py -> build/lib/tfds_defect_detection/downloader 2022-11-08T12:38:07,781 copying tfds_defect_detection/downloader/mvtec.py -> build/lib/tfds_defect_detection/downloader 2022-11-08T12:38:07,784 running egg_info 2022-11-08T12:38:07,930 writing tfds_defect_detection.egg-info/PKG-INFO 2022-11-08T12:38:07,935 writing dependency_links to tfds_defect_detection.egg-info/dependency_links.txt 2022-11-08T12:38:07,938 writing requirements to tfds_defect_detection.egg-info/requires.txt 2022-11-08T12:38:07,940 writing top-level names to tfds_defect_detection.egg-info/top_level.txt 2022-11-08T12:38:08,006 reading manifest file 'tfds_defect_detection.egg-info/SOURCES.txt' 2022-11-08T12:38:08,010 reading manifest template 'MANIFEST.in' 2022-11-08T12:38:08,017 writing manifest file 'tfds_defect_detection.egg-info/SOURCES.txt' 2022-11-08T12:38:08,091 /usr/local/lib/python3.7/dist-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. 2022-11-08T12:38:08,091 setuptools.SetuptoolsDeprecationWarning, 2022-11-08T12:38:08,156 installing to build/bdist.linux-armv7l/wheel 2022-11-08T12:38:08,157 running install 2022-11-08T12:38:08,219 running install_lib 2022-11-08T12:38:08,284 creating build/bdist.linux-armv7l 2022-11-08T12:38:08,285 creating build/bdist.linux-armv7l/wheel 2022-11-08T12:38:08,288 creating build/bdist.linux-armv7l/wheel/tfds_defect_detection 2022-11-08T12:38:08,289 copying build/lib/tfds_defect_detection/__init__.py -> build/bdist.linux-armv7l/wheel/tfds_defect_detection 2022-11-08T12:38:08,295 creating build/bdist.linux-armv7l/wheel/tfds_defect_detection/downloader 2022-11-08T12:38:08,296 copying build/lib/tfds_defect_detection/downloader/__init__.py -> build/bdist.linux-armv7l/wheel/tfds_defect_detection/downloader 2022-11-08T12:38:08,300 copying build/lib/tfds_defect_detection/downloader/visual_anomalies.py -> build/bdist.linux-armv7l/wheel/tfds_defect_detection/downloader 2022-11-08T12:38:08,304 copying build/lib/tfds_defect_detection/downloader/mvtec.py -> build/bdist.linux-armv7l/wheel/tfds_defect_detection/downloader 2022-11-08T12:38:08,307 copying build/lib/tfds_defect_detection/test.py -> build/bdist.linux-armv7l/wheel/tfds_defect_detection 2022-11-08T12:38:08,311 copying build/lib/tfds_defect_detection/utils.py -> build/bdist.linux-armv7l/wheel/tfds_defect_detection 2022-11-08T12:38:08,316 copying build/lib/tfds_defect_detection/data.py -> build/bdist.linux-armv7l/wheel/tfds_defect_detection 2022-11-08T12:38:08,320 running install_egg_info 2022-11-08T12:38:08,437 Copying tfds_defect_detection.egg-info to build/bdist.linux-armv7l/wheel/tfds_defect_detection-1.0.0-py3.7.egg-info 2022-11-08T12:38:08,457 running install_scripts 2022-11-08T12:38:08,628 creating build/bdist.linux-armv7l/wheel/tfds_defect_detection-1.0.0.dist-info/WHEEL 2022-11-08T12:38:08,633 creating '/tmp/pip-wheel-5vb7y_ta/tfds_defect_detection-1.0.0-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2022-11-08T12:38:08,639 adding 'tfds_defect_detection/__init__.py' 2022-11-08T12:38:08,643 adding 'tfds_defect_detection/data.py' 2022-11-08T12:38:08,646 adding 'tfds_defect_detection/test.py' 2022-11-08T12:38:08,649 adding 'tfds_defect_detection/utils.py' 2022-11-08T12:38:08,652 adding 'tfds_defect_detection/downloader/__init__.py' 2022-11-08T12:38:08,655 adding 'tfds_defect_detection/downloader/mvtec.py' 2022-11-08T12:38:08,658 adding 'tfds_defect_detection/downloader/visual_anomalies.py' 2022-11-08T12:38:08,663 adding 'tfds_defect_detection-1.0.0.dist-info/METADATA' 2022-11-08T12:38:08,665 adding 'tfds_defect_detection-1.0.0.dist-info/WHEEL' 2022-11-08T12:38:08,667 adding 'tfds_defect_detection-1.0.0.dist-info/top_level.txt' 2022-11-08T12:38:08,668 adding 'tfds_defect_detection-1.0.0.dist-info/RECORD' 2022-11-08T12:38:08,671 removing build/bdist.linux-armv7l/wheel 2022-11-08T12:38:08,871 Building wheel for tfds-defect-detection (setup.py): finished with status 'done' 2022-11-08T12:38:08,879 Created wheel for tfds-defect-detection: filename=tfds_defect_detection-1.0.0-py3-none-any.whl size=17209 sha256=428f0633dd758aca4947826fdf79b91ddbf56fb4cc237d08f214813d417b4fcc 2022-11-08T12:38:08,881 Stored in directory: /tmp/pip-ephem-wheel-cache-nleie4mb/wheels/36/ce/c1/a69c58c7a9435889c717868bf1e4123f744941f48aebeb3bcd 2022-11-08T12:38:08,909 Successfully built tfds-defect-detection 2022-11-08T12:38:08,917 Removed build tracker: '/tmp/pip-build-tracker-bbt9fe8y'