This study explores the challenges posed by pedestrian detection and occlusion in AR applications, employing a novel approach that utilizes RGB-D-based skeleton reconstruction to reduce the overhead of classical pedestrian detection algorithms during training. Furthermore, it is dedicated to addressing occlusion issues in pedestrian detection by using Azure Kinect for body tracking and integrating a robust occlusion management algorithm, significantly enhancing detection efficiency. In experiments, an average latency of 204 milliseconds was measured, and the detection accuracy reached an outstanding level of 97%. Additionally, this approach has been successfully applied in creating a simple yet captivating augmented reality game, demonstrating the practical application of the algorithm.
References
[1]
Dollar, P., Wojek, C., Schiele, B. and Perona, P. (2011) Pedestrian Detection: An Evaluation of the State of the Art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 743-761. https://doi.org/10.1109/TPAMI.2011.155
[2]
Dollár, P., Wojek, C., Schiele, B. and Perona, P. (2009) Pedestrian Detection: A Benchmark. 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, 20-25 June 2009, 304-311. https://doi.org/10.1109/CVPR.2009.5206631
[3]
Xiao, Y., Zhou, K., Cui, G., Jia, L., Fang, Z., Yang, X. and Xia, Q. (2021) Deep Learning for Occluded and Multi-Scale Pedestrian Detection: A Review. IET Image Processing, 15, 286-301. https://doi.org/10.1049/ipr2.12042
[4]
Zhang, S., Benenson, R., Omran, M., Hosang, J. and Schiele, B. (2016) How Far Are We from Solving Pedestrian Detection? 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, 27-30 June 2016, 1259-1267. https://doi.org/10.1109/CVPR.2016.141
[5]
Guo, X., Wang, C. and Qi, Y. (2017) Real-Time Augmented Reality with Occlusion Handling Based on RGBD Images. 2017 International Conference on Virtual Reality and Visualization (ICVRV), Zhengzhou, 21-22 October 2017, 298-302. https://doi.org/10.1109/ICVRV.2017.00069
[6]
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., et al. (2011) Real-Time Human Pose Recognition in Parts from Single Depth Images. CVPR 2011, Colorado Springs, CO, 20-25 June 2011, 1297-1304. https://doi.org/10.1109/CVPR.2011.5995316
[7]
Zhang, J., Chen, Z. and Tao, D. (2021) Towards High Performance Human Keypoint Detection. International Journal of Computer Vision, 129, 2639-2662. https://doi.org/10.1007/s11263-021-01482-8
[8]
Wang, J., Tan, S., Zhen, X., Xu, S., Zheng, F., He, Z. and Shao, L. (2021) Deep 3D Human Pose Estimation: A Review. Computer Vision and Image Understanding, 210, 103225. https://doi.org/10.1016/j.cviu.2021.103225
[9]
Wei, Q., Hu, W., Zhang, X. and Luo, G. (2007) Dominant Sets-Based Action Recognition Using Image Sequence Matching. 2007 IEEE International Conference on Image Processing, San Antonio, TX, 16 September-19 October 2007, VI-133-VI-136. https://doi.org/10.1109/ICIP.2007.4379539
[10]
Zharovskikh, A. (2020) The Role of Computer Vision in AR and VR. https://indatalabs.com/blog/computer-vision-ar-vr
[11]
Bandopadhyay, D. (2023) The Future of AR and Computer Vision: What to Expect. https://www.linkedin.com/pulse/future-ar-computer-vision-what-expect-debiprasad-bandopadhyay/
[12]
Barla, N. (2021) A Comprehensive Guide to Human Pose Estimation. https://www.v7labs.com/blog/human-pose-estimation-guide
[13]
Nilsen, T. and Looser, J. (2005) Tankwar-Tabletop War Gaming in Augmented Reality. 2nd International Workshop on Pervasive Gaming Applications, PerGames, 5.
[14]
Zhou, F., Duh, H. B. L. and Billinghurst, M. (2008) Trends in Augmented Reality Tracking, Interaction and Display: A Review of Ten Years of ISMAR. 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, Cambridge, 15-18 September 2008, 193-202. https://doi.org/10.1109/ISMAR.2008.4637362
[15]
Shah, M. M., Arshad, H. and Sulaiman, R. (2012) Occlusion in Augmented Reality. 2012 8th International Conference on Information Science and Digital Content Technology (ICIDT2012), 2, 372-378.
[16]
Macedo, M.C.D.F. and Apolinario, A.L. (2021) Occlusion Handling in Augmented Reality: Past, Present and Future. IEEE Transactions on Visualization and Computer Graphics, 29, 1590-1609. https://doi.org/10.1109/TVCG.2021.3117866
[17]
Alfakhori, M., Sardi Barzallo, J.S. and Coors, V. (2023) Occlusion Handling for Mobile AR Applications in Indoor and Outdoor Scenarios. Sensors, 23, 4245. https://doi.org/10.3390/s23094245
[18]
Lee, G. A., Billinghurst, M. and Kim, G. J. (2004) Occlusion Based Interaction Methods for Tangible Augmented Reality Environments. Proceedings of the 2004 ACM SIGGRAPH International Conference on Virtual Reality continuum and Its Applications in Industry, 419-426. https://doi.org/10.1145/1044588.1044680
