Development and research of a reinforcement learning method for acoustic diagnostics of industrial equipment

The actual problem of acoustic diagnostics of autonomously operating industrial equipment is investigated. An overview of existing approaches to acoustic diagnostics, including methods based on convolutional neural networks and learning algorithms with a teacher, is provided. Their limitations have been identified, such as the need to use large amounts of labeled data for training, poor adaptation to changing conditions, and the lack of a real-time decision-making mechanism. A new approach to acoustic diagnostics based on reinforcement learning methods is proposed, characterized by adaptability, high resistance to noise and the possibility of continuous learning in a dynamic environment. The proposed method for determining the state of equipment operability uses an approach based on the study of acoustic signals emitted by operating equipment. The method includes building a neural network, selecting audio recordings from open audio file libraries, and training the network using a reinforcement learning algorithm. The process of acoustic diagnostics of the state of serviceability/ malfunction of industrial equipment involves four stages: real-time recording of acoustic data of working equipment, extraction of signs of equipment condition, training with reinforcement of a neural network and making a decision on the serviceability / malfunction of the equipment. Based on tagged WAV audio files from open databases, an experiment was conducted to identify various states of the equipment: normal condition, initial stage of the defect, critical malfunction. The results showed classification accuracy from 89.7 % to 98.5 % and average response time from 0.5 to 0.7 seconds with low computing load (on average 36.5 % CPU and 509 MB RAM). Unlike the wellknown acoustic diagnostic systems based on teacher-learning algorithms for neural and convolutional neural networks on pre-marked datasets containing acoustic signals emitted by running equipment, the proposed approach implements the decomposition of the initial acoustic signals into spectral components. Each of these components is analyzed and provided with signs reflecting the state of serviceability or malfunction of the equipment. This approach allows you to: use reinforcement learning algorithms for strategic decision-making; reduce model training time by pre-selecting significant features; improve diagnostic accuracy; reduce computational load and hardware resource requirements. The developed algorithm can be used for continuous monitoring of equipment condition and predictive maintenance in autonomously functioning industrial systems. Its use will allow reliable and timely detection and classification of industrial equipment malfunctions. It is possible to refine the algorithm to meet the requirements for integration with the IoT infrastructure, increase resistance to external noise, and implement more advanced RL algorithms such as PPO. 

1. Vinogradenko A.M., Budko N.P. Adaptive control of technical condition of autonomous complex technical objects on the basis of intelligent technologies. Electrostatic gyroscope in spacecraft attitude reference systems. T-Comm, 2020, vol. 14, no. 1, pp. 25–35. (in Russian). https://doi.org/10.36724/2072-8735-2020-14-1-25-35

2. Bogatyrev V.A., Bogatyrev S.V., Bogatyrev A.V. Assessment of the readiness of a computer system for timely servicing of requests when combined with information recovery of memory after failures. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2023, vol. 23, no. 3, pp. 608–617. (in Russian). https://doi.org/10.17586/2226-1494-2023-23-3-608-617

3. Bogatyrev V., Vinokurova M. Control and safety of operation of duplicated computer systems. Communications in Computer and Information Science, 2017, vol. 700, pp. 331–342. https://doi.org/10.1007/978-3-319-66836-9_28

4. Bogatyrev V.A. Exchange of duplicated computing complexes in fault-tolerant systems. Automatic Control and Computer Sciences, 2011, vol. 45, no. 5, pp. 268–276. https://doi.org/10.3103/s014641161105004x

5. Martiugov A.S., Ershov E.V., Vinogradova L.N., Varfolomeev I.A. Diagnostics of industrial equipment using acoustic testing. Proc. of the Optical-Electronic Devices and Instruments in Image Recognition and Processing Systems, 2021, pp. 172–174. (in Russian)

6. Verzun N.A., Kolbanev M.O., Salieva A.R. Multi-agent ensemble algorithm for acoustic recognition of malfunctions of autonomous technological equipment. Information and Control Systems, 2025, no. 3, pp. 14–24. (in Russian). https://doi.org/10.31799/1684-8853-2025-3-14-24

7. Shchegolkov M.V., Zinkin S.A. Overview of the main model-free reinforcement learning approaches. Proc. of the Energy and Automation in Modern Society, 2024, pp. 91–95. (in Russian)

8. Ye L., Ma X., Wen C. Rotating machinery fault diagnosis method by combining time-frequency domain features and CNN knowledge transfer. Sensors, 2021, vol. 21, no. 24, pp. 8168. https://doi.org/10.3390/s21248168

9. Shao S., McAleer S., Yan R., Baldi P. Highly accurate machine fault diagnosis using deep transfer learning. IEEE Transactions on Industrial Informatics, 2019, vol. 15, no. 4, pp. 2446–2455. https://doi.org/10.1109/tii.2018.2864759

10. Souza R.M., Nascimento E.G.S., Miranda U.A., Silva W.J.D., Lepikson H.A.. Deep learning for diagnosis and classification of faults in industrial rotating machinery. Computers and Industrial Engineering, 2021, vol. 153, pp. 107060. https://doi.org/10.1016/j.cie.2020.107060

11. Lyu P., Zhang K., Yu W., Wang B., Liu C. A novel RSG-based intelligent bearing fault diagnosis method for motors in high-noise industrial environment. Advanced Engineering Informatics, 2022, vol. 52, pp. 101564. https://doi.org/10.1016/j.aei.2022.101564

12. Zhang J., Koppel A., Bedi A.S., Szepesvari C., Wang M., Variational policy gradient method for reinforcement learning with general utilities. arXiv, 2020, arXiv:2007.02151. https://doi.org/10.48550/arXiv.2007.02151

13. Chen D., Peng P., Huang T., Tian Y. Deep reinforcement learning with spiking Q-learning. arXiv, 2022, arXiv:2201.09754. https://doi.org/10.48550/arXiv.2201.09754

14. Verzun N.A., Kolbanev M.O., Salieva A.R. Analysis learning prospects of smart autonomous logistics systems based on value function optimization. LETI Transactions on Electrical Engineering & Computer Science, 2024, vol. 17, no. 10, pp. 28–39. (in Russian). https://doi.org/10.32603/2071-8985-2024-17-10-28-39

15. Tama B.A., Vania M., Lee S., Lim S. Recent advances in the application of deep learning for fault diagnosis of rotating machinery using vibration signals. Artificial Intelligence Review, 2023, vol. 56, no. 5, pp. 4667–4709. https://doi.org/10.1007/s10462-022-10293-3

16. Wang R., Zhan X., Bai H., Dong E., Cheng Z., Jia X. A review of fault diagnosis methods for rotating machinery using infrared thermography. Micromachines, 2022, vol. 13, no. 10, pp. 1644. https://doi.org/10.3390/mi13101644

17. Ramaswamy A., Hüllermeier E. Deep Q-Learning: theoretical insights from an asymptotic analysis. arXiv, 2020, arXiv:2008.10870. https://doi.org/10.48550/arXiv.2008.10870

18. Hansen N., Su H., Wang X. Stabilizing Deep Q-Learning with ConvNets and vision transformers under data augmentation. Proc. of the 35th International Conference on Neural Information Processing Systems, 2021, pp. 3680–3693.

19. Haq A.S., Nasrun M., Setianingsih C., Murti M.A. Speech recognition implementation using MFCC and DTW algorithm for home automation. Proc. of the International Conference on Electrical Engineering Computer Science and Informatics, 2020, vol. 7, pp. 78–85. https://doi.org/10.11591/eecsi.v7.2041

20. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction. Bradford Books, 2018, 552 p.

21. Das O., Das D.B., Birant D. Machine learning for fault analysis in rotating machinery: A comprehensive review. Heliyon, 2023, vol. 9, no. 6, pp. e17584. https://doi.org/10.1016/j.heliyon.2023.e17584

22. Berdnikova A.A., Kolbanev M.O, Verzun N.A., Salieva A.R. Acoustic system for diagnostics of industrial equipment faults based on reinforcement learning (ASD-OP). Certificate of state registration of the computer program RU 2025619237. 2025. (in Russian)

23. Moharam M.H., Hany O., Hany A., Mahmoud A., Mohamed M., Saeed S. Anomaly detection using machine learning and adopted digital twin concepts in radio environments. Scientific Reports, 2025, vol. 15, pp. 18352. https://doi.org/10.1038/s41598-025-02759-5

24. Purohit H.P., Tanabe R., Ichige K., Endo T., Nikaido Y., Suefusa K., Kawaguchi Y. MIMII Dataset: sound dataset for malfunctioning industrial machine investigation and inspection. Proc. of the Detection and Classification of Acoustic Scenes and Events 2019 Workshop (DCASE2019), 2019, pp. 209–213. https://doi.org/10.33682/m76f-d618

25. Koizumi Y., Kawaguchi Y., Imoto K., Nakamura T., Nikaido Y., Tanabe R., Purohit H., Suefusa K., Endo T., Yasuda M., Harada N. Description and discussion on DCASE2020 challenge Task2: unsupervised anomalous sound detection for machine condition monitoring. arXiv, 2020, arXiv:2006.05822. https://doi.org/10.48550/arXiv.2006.05822