Although compressive measurements save data storage
and bandwidth usage, they are difficult to be used directly for target tracking
and classification without pixel reconstruction. This is because the Gaussian
random matrix destroys the target location information in the original video
frames. This paper summarizes our research effort on target tracking and
classification directly in the compressive measurement domain. We focus on one
particular type of compressive measurement using pixel subsampling. That is,
original pixels in video frames are randomly subsampled. Even in such a special
compressive sensing setting, conventional trackers do not work in a
satisfactory manner. We propose a deep learning approach that integrates YOLO
(You Only Look Once) and ResNet (residual network) for multiple target tracking
and classification. YOLO is used for multiple target tracking and ResNet is for
target classification. Extensive experiments using short wave infrared (SWIR),
mid-wave infrared (MWIR), and long-wave infrared (LWIR) videos demonstrated the
efficacy of the proposed approach even though the training data are very
scarce.
References
[1]
Li, X., Kwan, C., Mei, G. and Li, B. (2006) A Generic Approach to Object Matching and Tracking. In: Campilho, A. and Kamel, M.S., Eds., Image Analysis and Recognition. ICIAR 2006. Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 839-849. https://doi.org/10.1007/11867586_76
[2]
Zhou, J. and Kwan, C. (2018) Tracking of Multiple Pixel Targets Using Multiple Cameras. In: Huang, T., Lv, J., Sun, C. and Tuzikov, A., Eds., Advances in Neural Networks. Lecture Notes in Computer Science, Springer, Cham, 484-493.
https://doi.org/10.1007/978-3-319-92537-0_56
[3]
Zhou, J. and Kwan, C. (2018) Anomaly Detection in Low Quality Traffic Monitoring Videos Using Optical Flow. Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, 106490F.
[4]
Kwan, C., Zhou, J., Wang, Z. and Li, B. (2018) Efficient Anomaly Detection Algorithms for Summarizing Low Quality Videos. Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, 1064906. https://doi.org/10.1117/12.2303764
[5]
Kwan, C., Chou, B. and Kwan, L. M. (2018) A Comparative Study of Conventional and Deep Learning Target Tracking Algorithms for Low Quality Videos. In: Huang, T., Lv, J., Sun, C. and Tuzikov, A., Eds., Advances in Neural Networks. Lecture Notes in Computer Science, Springer, Cham, 521-531.
https://doi.org/10.1007/978-3-319-92537-0_60
[6]
Kwan, C., Yin, J. and Zhou, J. (2018) The Development of a Video Browsing and Video Summary Review Tool. Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, 1064907. https://doi.org/10.1117/12.2303654
[7]
Zhao, Z., Chen, H., Chen, G., Kwan, C. and Li, X.R. (2006) IMM-LMMSE Filtering Algorithm for Ballistic Target Tracking with Unknown Ballistic Coefficient. Proceedings of SPIE, Volume 6236, Signal and Data Processing of Small Targets.
https://doi.org/10.1117/12.665760
[8]
Zhao, Z., Chen, H., Chen, G., Kwan, C. and Li, X.R. (2006) Comparison of Several Ballistic Target Tracking Filters. Proceedings of American Control Conference, Minneapolis, MN, 14-16 June 2006, 2197-2202.
[9]
Candes, E.J. and Wakin, M.B. (2008) An Introduction to Compressive Sampling. IEEE Signal Processing Magazine, 25, 21-30.
https://doi.org/10.1109/MSP.2007.914731
[10]
Kwan, C., Chou, B., Echavarren, A., Budavari, B., Li, J. and Tran, T. (2018) Compressive Vehicle Tracking Using Deep Learning. IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York City, 8-10 November 2018, 51-56. https://doi.org/10.1109/UEMCON.2018.8796778
[11]
Tropp, J.A. (2004) Greed Is Good: Algorithmic Results for Sparse Approximation. IEEE Transactions on Information Theory, 50, 2231-2242.
https://doi.org/10.1109/TIT.2004.834793
[12]
Yang, J. and Zhang, Y. (2011) Alternating Direction Algorithms for L1-Problems in Compressive Sensing. SIAM Journal on Scientific Computing, 33, 250-278.
https://doi.org/10.1137/090777761
[13]
Dao, M., Kwan, C., Koperski, K. and Marchisio, G. (2017) A Joint Sparsity Approach to Tunnel Activity Monitoring Using High Resolution Satellite Images. 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference, New York, 19-21 October 2017, 322-328.
https://doi.org/10.1109/UEMCON.2017.8249061
[14]
Zhou, J., Ayhan, B., Kwan, C. and Tran, T. (2018) ATR Performance Improvement Using Images with Corrupted or Missing Pixels. Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, 106490E.
[15]
Applied Research LLC (2017) Phase 1 Final Report.
[16]
Kwan, C., Chou, B., Yang, J. and Tran, T. (2019) Target Tracking and Classification Directly in Compressive Measurement for Low Quality Videos. Pattern Recognition and Tracking XXX (Conference SI120). https://doi.org/10.1117/12.2518496
[17]
Kwan, C., Gribben, D. and Tran, T. (2019) Multiple Human Objects Tracking and Classification Directly in Compressive Measurement Domain for Long Range Infrared Videos. IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York City.
[18]
Kwan, C., Chou, B., Yang, J., and T. Tran, T. (2019) Deep Learning Based Target Tracking and Classification Directly in Compressive Measurement for Low Quality Videos. Signal & Image Processing: An International Journal (SIPIJ).
[19]
Kwan, C., Gribben, D. and Tran, T. (2019) Tracking and Classification of Multiple Human Objects Directly in Compressive Measurement Domain for Low Quality Optical Videos. IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York City.
[20]
Kwan, C., Chou, B., Yang, J., Rangamani, A., Tran, T., Zhang, J. and Etienne-Cummings, R. (2019) Target Tracking and Classification Directly Using Compressive Sensing Camera for SWIR Videos. Signal, Image, and Video Processing, 13, 1629-1637. https://doi.org/10.1007/s11760-019-01506-4
[21]
Kwan, C., Chou, B., Yang, J., Rangamani, A., Tran, T., Zhang, J. and Etienne-Cummings, R. (2019) Target Tracking and Classification Using Compressive Measurements of MWIR and LWIR Coded Aperture Cameras. Journal Signal and Information Processing, 10, 73-95. https://doi.org/10.4236/jsip.2019.103006
[22]
Kwan, C., Chou, B., Yang, J., Rangamani, A., Tran, T., Zhang, J. and Etienne-Cummings, R., (2019) Deep Learning based Target Tracking and Classification for Low Quality Videos Using Coded Aperture Camera. Sensors, 19, 3702.
https://doi.org/10.3390/s19173702
[23]
Yang, M.H., Zhang, K. and Zhang, L. (2012) Real-Time Compressive Tracking. In European Conference on Computer Vision.
[24]
He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. 2016 Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, 27-30 June 2016, 770-778. https://doi.org/10.1109/CVPR.2016.90
[25]
Redmon, J. and Farhadi, A. (2018) YOLOv3: An Incremental Improvement.
[26]
Ren S., He, K., Girshick, R. and Sun, J. (2015) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In: Advances in Neural Information Processing Systems, 1-9.
[27]
Kwan, C., Chou, B., Yang, J., Budavari, B., and Tran, T. (2019) Compressive Object Tracking and Classification Using Deep Learning for Infrared Videos. Pattern Recognition and Tracking XXX (Conference SI120).
https://doi.org/10.1117/12.2518490
[28]
Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O. and Torr, P. (2016) Staple: Complementary Learners for Real-Time Tracking. 2016 Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, 27-30 June 2016, 1401-1409.
https://doi.org/10.1109/CVPR.2016.156
[29]
Stauffer, C. and Grimson, W.E.L. (1999) Adaptive Background Mixture Models for Real-Time Tracking, Computer Vision and Pattern Recognition. IEEE Computer Society Conference, 2, 2246-2252.
