ntvНаучно-технический вестник информационных технологий, механики и оптикиScientific and Technical Journal of Information Technologies, Mechanics and Optics2226-14942500-0373Университет ИТМО10.17586/2226-1494-2024-24-6-972-981ntv-403Research ArticleКОМПЬЮТЕРНЫЕ СИСТЕМЫ И ИНФОРМАЦИОННЫЕ ТЕХНОЛОГИИCOMPUTER SCIENCEОбнаружение дефектов твердых поверхностей при ограниченном объеме данных на основе SSD-детектора и сиамских сетейSurface defect detection with limited data based on SSD detector and Siamese networkshttps://orcid.org/0009-0000-5352-1852НовгородцевН. П.NovgorodcevN. P.

Новгородцев Никита Павлович - студент, 

Санкт-Петербург, 197101

Nikita P. Novgorodcev - Student,

Saint Petersburg, 197101

novgorodcevnp@gmail.comhttps://orcid.org/0009-0003-7555-282XБатуринаК. А.BaturinaK. A.

Батурина Ксения Александровна - студент, лаборант,

Санкт-Петербург, 197101

Kseniia A. Baturina - Student, Laboratory Assistant, 

Saint Petersburg, 197101

kseniya.baturina@mail.ruhttps://orcid.org/0000-0002-5309-2207ЕфимоваВ. А.EfimovaV. A.

Ефимова Валерия Александровна - кандидат технических наук,ассистент,

Санкт-Петербург, 197101

Valeria A. Efimova - PhD, Assistant,

Saint Petersburg, 197101

valeryefimova@gmail.comhttps://orcid.org/0000-0002-2723-2077ШалытоА. А.ShalytoA. A.

Шалыто Анатолий Абрамович - доктор технических наук, профессор, главный научный сотрудник, профессор,

Санкт-Петербург, 197101

Anatoly A. Shalyto - D.Sc., Full Professor, Chief Researcher,

Saint Petersburg, 197101

shalyto@mail.ifmo.ruУниверситет ИТМОРоссияITMO UniversityRussian Federation202429122024246972981Copyright © Новгородцев Н.П., Батурина К.А., Ефимова В.А., Шалыто А.А., 20242024Новгородцев Н.П., Батурина К.А., Ефимова В.А., Шалыто А.А.Novgorodcev N.P., Baturina K.A., Efimova V.A., Shalyto A.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ntv.elpub.ru/jour/article/view/403Введение

Введение. Представлен алгоритм решения задачи обнаружения дефектов твердых поверхностей при обучении на нулевом или малом числе примеров, который решает проблему ограниченного объема данных. Усовершенствуется существующая методология обнаружения дефектов методом с использованием машинного зрения.

Метод

Метод. Предложен гибридный подход, сочетающий преимущества SSD-детектора и сиамских нейронных сетей. SSD-детектор позволяет извлекать векторные представления признаков из изображений, а сиамские нейронные сети применяются для построения пространства извлеченных признаков.

Основные результаты

Основные результаты. Показано, что новый подход обладает высокой точностью как на известных, так и на не встречавшихся ранее в обучающей выборке дефектах. По результатам тестирования на 7 различных наборах данных представленный алгоритм продемонстрировал хорошие возможности в сценариях с ограниченным числом примеров для обучения.

Обсуждение

Обсуждение. Сравнительный анализ с существующими моделями показал высокую производительность предлагаемого алгоритма и его потенциал как инновационного и эффективного решения задач универсального обнаружения дефектов твердых поверхностей.

This study presents an algorithm for the problem of detecting defects on hard surfaces when trained with zero or a small number of examples, addressing the challenge of limited data availability. The existing defect detection methodology using machine vision is enhanced. A hybrid approach is proposed, combining the strengths of the SSD detector and Siamese Neural Networks (SNN). The SSD detector extracts feature vector representations from images, while the SNNs are used to construct the feature space. The new approach demonstrates high accuracy in detecting both known and previously unseen defects in the training dataset. Based on testing across seven different datasets, the model showed good performance in scenarios with a limited number of training examples. A comparative analysis with existing models highlights the high performance of the proposed approach and its potential as an innovative and effective solution for the universal detection of defects on hard surfaces.

компьютерное зрениеобнаружение дефектовобучение с нулевым количеством примеровобнаружение объектовсиамские сетиcomputer visiondefects detectionzero-shot learningobject detectionSiamese networksReferencesKamiya N., Zhou X., Chen H., Muramatsu C., Hara T., Fujita H. Model-based approach to recognize the rectus abdominis muscle in CT images // IEICE Transactions on Information and Systems. 2013. V. E96D. N 4. P. 869–871. https://doi.org/10.1587/transinf.e96.d.869Kamiya N., Zhou X., Chen H., Muramatsu C., Hara T., Fujita H. Model-based approach to recognize the rectus abdominis muscle in CT images. IEICE Transactions on Information and Systems, 2013, vol. E96D, no. 4, pp. 869–871. https://doi.org/10.1587/transinf.e96.d.869Bai T., Gao J., Yang J., Yao D. A study on railway surface defects detection based on machine vision // Entropy. 2021. V. 23. N 11. P. 1437. https://doi.org/10.3390/e23111437Bai T., Gao J., Yang J., Yao D. A study on railway surface defects detection based on machine vision. Entropy, 2021, vol. 23, no. 11, pp. 1437. https://doi.org/10.3390/e23111437Saberironaghi A., Ren J., El-Gindy M. Defect detection methods for industrial products using deep learning techniques: a review // Algorithms. 2023. V. 16. N 2. P. 95. https://doi.org/10.3390/a16020095Saberironaghi A., Ren J., El-Gindy M. Defect detection methods for industrial products using deep learning techniques: a review. Algorithms, 2023, vol. 16, no. 2, pp. 95. https://doi.org/10.3390/a16020095Srividhya R., Shanmugapriya K., SindhuPriya K. Automatic detection of surface defects in industrial materials based on image processing // International Journal of Engineering & Technology. 2018. V. 7. N 3.34. P. 61–64. https://doi.org/10.14419/ijet.v7i3.34.18717Srividhya R., Shanmugapriya K., SindhuPriya K. Automatic detection of surface defects in industrial materials based on image processing. International Journal of Engineering & Technology, 2018, vol. 7, no. 3.34, pp. 61–64. https://doi.org/10.14419/ijet.v7i3.34.18717Guijo D., Onofre V., Del Bimbo G., Mugel S., Estepa D., De Carlos X., Adell A., Lojo A., Bilbao J., Orus R. Quantum artificial vision for defect detection in manufacturing // arXiv. 2022. arXiv:2208.04988. https://doi.org/10.48550/arXiv.2208.04988Guijo D., Onofre V., Del Bimbo G., Mugel S., Estepa D., De Carlos X., Adell A., Lojo A., Bilbao J., Orus R. Quantum artificial vision for defect detection in manufacturing. arXiv, 2022. arXiv:2208.04988. https://doi.org/10.48550/arXiv.2208.04988Prajwala N.B. Defect detection in pharma pills using image processing // International Journal of Engineering & Technology. 2018. V. 7. N 3.3. P. 102–106. https://doi.org/10.14419/ijet. v7i3.3.14497Prajwala N.B. Defect detection in pharma pills using image processing. International Journal of Engineering & Technology, 2018, vol. 7, no. 3.3, pp. 102–106. https://doi.org/10.14419/ijet.v7i3.3.14497Liu W., Anguelov D., Erhan D., Szegedy C., Reed S., Fu C.-Y., Berg A.C. SSD: Single shot multibox detector // Lecture Notes in Computer Science. 2016. V. 9905. P. 21–37. https://doi.org/10.1007/978-3-319-46448-0_2Liu W., Anguelov D., Erhan D., Szegedy C., Reed S., Fu C.-Y., Berg A.C. SSD: Single shot multibox detector. Lecture Notes in Computer Science, 2016, vol. 9905, pp. 21–37. https://doi.org/10.1007/978-3-319-46448-0_2Bromley J., Guyon I., LeCun Y., Säckinger E., Shah R. Signature verification using a “Siamese” time delay neural network // Advances in Neural Information Processing Systems. 1993. V. 6.Bromley J., Guyon I., LeCun Y., Säckinger E., Shah R. Signature verification using a «Siamese» time delay neural network. Advances in Neural Information Processing Systems, 1993, vol. 6. 9. Song K.-C., Hu S., Yan Y. Automatic recognition of surface defects on hot-rolled steel strip using scattering convolution network. Journal of Computational Information Systems, 2014, vol. 10, no. 7, pp. 3049– 3055.Song K.-C., Hu S., Yan Y. Automatic recognition of surface defects on hot-rolled steel strip using scattering convolution network // Journal of Computational Information Systems. 2014. V. 10. N 7. P. 3049–3055.Kodytek P., Bodzas A., Bilik P. Supporting data for Deep Learning and Machine Vision based approaches for automated wood defect detection and quality control. Zenodo, 2015. Available at: https://zenodo.org/records/4694695 (accessed: 30.10.2024)Kodytek P., Bodzas A., Bilik P. Supporting data for Deep Learning and Machine Vision based approaches for automated wood defect detection and quality control // Zenodo. 2015 [Электронный ресурс]. URL: https://zenodo.org/records/4694695 (дата обращения: 30.10.2024)Bergmann P., Batzner K., Fauser M., Sattlegger D., Steger C. The MVTec anomaly detection dataset: a comprehensive real-world dataset for unsupervised anomaly detection. International Journal of Computer Vision, 2021, vol. 129, no. 4, pp. 1038–1059. https://doi.org/10.1007/s11263-020-01400-4Bergmann P., Batzner K., Fauser M., Sattlegger D., Steger C. The MVTec anomaly detection dataset: a comprehensive real-world dataset for unsupervised anomaly detection // International Journal of Computer Vision. 2021. V. 129. N 4. P. 1038–1059. https://doi.org/10.1007/s11263-020-01400-4Nagy A.M., Czúni L. Detecting object defects with fusioning convolutional siamese neural networks. Proc. of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP). V. 5, 2021, pp. 157– 163. https://doi.org/10.5220/0010263301570163Nagy A.M., Czúni L. Detecting object defects with fusioning convolutional siamese neural networks // Proc. of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP). V. 5. 2021. P. 157–163. https://doi.org/10.5220/0010263301570163Karmakar S., Banerjee A., Gidde P., Saurav S., Singh S. Convolutional ensembling based few-shot defect detection technique. Proc. of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP ‘22), 2022, pp. 1–7. https://doi.org/10.1145/3571600.3571607Karmakar S., Banerjee A., Gidde P., Saurav S., Singh S. Convolutional ensembling based few-shot defect detection technique // Proc. of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP ‘22). 2022. P. 1–7. https://doi.org/10.1145/3571600.3571607Lv Q., Song Y. Few-shot learning combine attention mechanismbased defect detection in bar surface. ISIJ International, 2019, vol. 59, no. 6, pp. 1089–1097. https://doi.org/10.2355/isijinternational.isijint-2018-722Lv Q., Song Y. Few-shot learning combine attention mechanismbased defect detection in bar surface // ISIJ International. 2019. V. 59. N 6. P. 1089–1097. https://doi.org/10.2355/isijinternational.isijint-2018-722Schlagenhauf T., Yildirim F., Brückner B. Siamese basis function networks for data-efficient defect classification in technical domains. Lecture Notes in Computer Science, 2023, vol. 13765, pp. 71–92. https://doi.org/10.1007/978-3-031-26236-4_7Schlagenhauf T., Yildirim F., Brückner B. Siamese basis function networks for data-efficient defect classification in technical domains // Lecture Notes in Computer Science. 2023. V. 13765. P. 71–92. https://doi.org/10.1007/978-3-031-26236-4_7Nagy A.M., Czúni L. Classification and fast few-shot learning of steel surface defects with randomized network. Applied Sciences, 2022, vol. 12, no. 8, pp. 3967. https://doi.org/10.3390/app12083967Nagy A.M., Czúni L. Classification and fast few-shot learning of steel surface defects with randomized network // Applied Sciences. 2022. V. 12. N 8. P. 3967. https://doi.org/10.3390/app12083967Cao Y., Xu X., Sun C., Cheng Y., Du Z., Gao L., Shen W. Segment any anomaly without training via hybrid prompt regularization. arXiv, 2023. arXiv:2305.10724. https://doi.org/10.48550/arXiv.2305.10724Cao Y., Xu X., Sun C., Cheng Y., Du Z., Gao L., Shen W. Segment any anomaly without training via hybrid prompt regularization // arXiv. 2023. arXiv:2305.10724. https://doi.org/10.48550/arXiv.2305.10724Cao Y., Xu X., Sun C., Cheng Y., Du Z., Gao L., Shen W. Segment any anomaly without training via hybrid prompt regularization // arXiv. 2023. arXiv:2305.10724. https://doi.org/10.48550/arXiv.2305.10724

The authors declare that there are no conflicts of interest present.