The impact of adversarial attacks on a computer vision models perception of images Set intersection protocol with privacy preservation

   Advances in computer vision have led to the development of powerful models capable of accurately recognizing and interpreting visual information in various fields of knowledge. However, these models are increasingly vulnerable to adversarial attacks – deliberate manipulations of input data designed to mislead the machine-learning model and produce incorrect recognition results. This article presents the results of an investigation into the impact of various types of adversarial attacks on the ResNet50 model in image classification and clustering tasks. Various types of adversarial attacks have been investigated: Fast Gradient Sign Method, Basic Iterative Method, Projected Gradient Descent, Carlini&Wagner, Elastic-Net Attacks to Deep Neural Networks, Expectation Over Transformation Projected Gradient Descent, and jitter-based attacks. The Gradient-Weighted Class Activation Mapping (Grad-CAM) method was used to visualize the attention areas of the model. The t-SNE algorithm was applied to visualize clusters in the feature space. Attack robustness was assessed by attack success rate using k-Nearest Neighbors algorithm and Hierarchical Navigable Small World algorithms with different similarity metrics. Significant differences in the effects of attacks on the internal representations of the model and areas of focus have been identified. It is shown that iterative attack methods cause significant changes in the feature space and significantly affect Grad-CAM visualizations, whereas simple attacks have less impact. The high sensitivity of most clustering algorithms to perturbations has been established. The metric of the inner product showed the greatest stability among the studied approaches. The results obtained indicate the dependence of the stability of the model on the attack parameters and the choice of similarity metrics, which is manifested in the peculiarities of the formation of cluster structures. The observed feature-space redistributions under targeted attacks suggest avenues for further optimizing clustering algorithms to enhance the resilience of computer-vision systems.

1. Liu A. Guo J., Wang J., Liang S., Tao R., Zhou W., Liu C., Liu X., Tao D. X-adv: Physical adversarial object attacks against x-ray prohibited item detection. arXiv, 2023, arXiv:2302.09491. doi: 10.48550/arXiv.2302.09491

2. Goodfellow I.J., Shlens J., Szegedy C. Explaining and harnessing adversarial examples. arXiv, 2015, arXiv:1412.6572. doi: 10.48550/arXiv.1412.6572

3. Madry A., Makelov A., Schmidt L., Tsipras D., Vladu A. Towards deep learning models resistant to adversarial attacks. arXiv, 2019, arXiv:1706.06083. doi: 10.48550/arXiv.1706.06083

4. Carlini N., Wagner D. Towards evaluating the robustness of neural networks. Proc. of the IEEE Symposium on Security and Privacy (SP), 2017, pp. 39–57. doi: 10.1109/SP.2017.49

5. Qian Y., He S., Zhao C., Sha J. Wang W., Wang B. Lea2: A lightweight ensemble adversarial attack via non-overlapping vulnerable frequency regions. Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 4487–4498. doi: 10.1109/iccv51070.2023.00416

6. Schlarmann C., Singh N.D., Croce F., Hein M. Robust CLIP: unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models. Proc. of the 41^st International Conference on Machine Learning, 2024, no. 1779. pp. 43684–43704.

7. He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778. doi: 10.1109/CVPR.2016.90

8. Liu X., Hu J., Yang Q., Jiang M., He J., Fang H. A divide-and-conquer reconstruction method for defending against adversarial example attacks. Visual Intelligence, 2024, vol. 2, pp. 30. doi: 10.1007/s44267-024-00061-y

9. Zhang J., Wu W., Huang J., Huang Y., Wang W., Su Y., Lyu M. Improving adversarial transferability via neuron attribution-based attacks. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 14973–14982. doi: 10.1109/CVPR52688.2022.01457

10. Kurakin A., Goodfellow I., Bengio S. Adversarial examples in the physical world. Artificial Intelligence Safety and Security, 2018, pp. 14. doi: 10.1201/9781351251389-8

11. Chen P.-Y., Sharma Y., Zhang H., Yi J. & Hsieh C.-J. Ead: Elastic-net attacks to deep neural networks via adversarial examples. Proc. of the 32^nd AAAI Conference on Artificial Intelligence, 2018, vol. 32, no. 1, pp. 10–17. doi: 10.1609/aaai.v32i1.11302

12. Zimmermann R.S. Comment on “adv-bnn: Improved adversarial defense through robust bayesian neural network”. arXiv, 2019, arXiv:1907.00895. doi: 10.48550/arXiv.1907.00895

13. Schwinn L., Raab R., Nguyen A., Zanca D., Eskofier B. Exploring misclassifications of robust neural networks to enhance adversarial attacks. Applied Intelligence, 2023, vol. 53, no. 17, pp. 19843–19859. doi: 10.1007/s10489-023-04532-5

14. Selvaraju R.R., Cogswell M., Das A., Vedantam R., Parikh D., Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision, 2020, vol. 128, no. 2, pp. 336–359. doi: 10.1007/s11263-019-01228-7

15. van der Maaten L., Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, vol. 9, pp. 2579–2605.

16. Fix E., Hodges J. Discriminatory Analysis. Nonparametric Discrimination: Consistency Properties. USAF School of Aviation Medicine, 1951, 44 p.

17. Malkov Y.A., Yashunin D.A. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, vol. 42, no. 4, pp. 824–836. doi: 10.1109/TPAMI.2018.2889473

18. ImageNet Large Scale Visual Recognition Challenge (ILSVRC). Stanford Vision Lab, Stanford University, Princeton University. ImageNet Data [Электронный ресурс]. Available at: https://www.image-net.org/download.php. (accessed: 03. 03. 2025).