Method for discovering spatial arm positions with depth sensor data at low-performance devices
https://doi.org/10.17586/2226-1494-2022-22-2-410-414
Abstract
A method of arm aiming direction estimation for low performance Internet of Things devices is proposed. It uses Human Pose Estimation (HPE) algorithms for retrieving human skeleton key points. Having these key points, arm aiming directions model is calculated. Two well-known HPE methods (PoseNet and OpenPose) are examined. These algorithms have been tested and compared by the average angle of error. The system includes a Raspberry Pi 4B singleboard computer and an Intel RealSense D435i depth sensor. The developed approach may be utilized in “smart home” gesture control systems.
About the Authors
D. S. Medvedev
ITMO University
Russian Federation
Russian Federation
Dmitrii S. Medvedev — Engineer
Saint Petersburg, 197101
A. D. Ignatov
Federal Reserch Center “Computer Science and Control” of RAS
Russian Federation
Russian Federation
Andrei D. Ignatov — Research Engineer
Moscow, 119333
sc 57205407925
References
1. Shmatkov V.N., Bąkowski P., Medvedev D.S., Korzukhin S.V., Golendukhin D.V., Spynu S.F., Mouromtsev D.I. Interaction with Internet of Things devices by voice control. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2019, vol. 19, no. 4, pp. 714–721. (in Russian). https://doi.org/10.17586/2226-1494-2019-19-4-714-721
2. Chen W., Yu C., Tu C., Lyu Z., Tang J., Ou S., Fu Y., Xue Z. A survey on hand pose estimation with wearable sensors and computer-visionbased methods. Sensors, 2020, vol. 20, no. 4, pp. 1074. https://doi.org/10.3390/s20041074
3. Shotton J., Fitzgibbon A., Cook M., Sharp T., Finocchio M., Moore R., Kipman A., Blake A. Real-time human pose recognition in parts from single depth images. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 1297– 1304. https://doi.org/10.1109/CVPR.2011.5995316
4. Hernández-Vela A., Zlateva N., Marinov A., Reyes M., Radeva P., Dimov D., Escalera S. Graph cuts optimization for multi-limb human segmentation in depth maps. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 726–732. https://doi.org/10.1109/CVPR.2012.6247742
5. Ye M., Wang X., Yang R., Ren L., Pollefeys M. Accurate 3D pose estimation from a single depth image. Proc. of the IEEE International Conference on Computer Vision, 2011, pp. 731–738. https://doi.org/10.1109/ICCV.2011.6126310
6. Shafaei A., Little J.J. Real-time human motion capture with multiple depth cameras. Proc. of the 13th Conference on Computer and Robot Vision (CRV), 2016, pp. 24–31. https://doi.org/10.1109/CRV.2016.25
7. Marin-Jimenez M.J., Romero-Ramirez F.J., Muñoz-Salinas R., Medina-Carnicer R. 3D human pose estimation from depth maps using a deep combination of poses. Journal of Visual Communication and Image Representation, 2018, vol. 55, pp. 627–639. https://doi.org/10.1016/j.jvcir.2018.07.010
8. Cao Z., Hidalgo G., Simon T., Wei S.-E., Sheikh Y. OpenPose: Realtime Multi-person 2D pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, vol. 43, no. 1, pp. 8765346. https://doi.org/10.1109/ TPAMI.2019.2929257
9. Kendall A., Grimes M., Cipolla R. PoseNet: A convolutional network for real-time 6-dof camera relocalization. Proc. of the 15th IEEE International Conference on Computer Vision (ICCV), 2015, pp. 2938–2946. https://doi.org/10.1109/ICCV.2015.336
Review
For citations:
Medvedev D.S., Ignatov A.D. Method for discovering spatial arm positions with depth sensor data at low-performance devices. Scientific and Technical Journal of Information Technologies, Mechanics and Optics. 2022;22(2):410-414. (In Russ.) https://doi.org/10.17586/2226-1494-2022-22-2-410-414
Views:
1
Email this article 