Pixel-wise Code Exposure (PCE) camera is one type of compressive sensing
camera that has low power consumption and high compression ratio. Moreover,
a PCE camera can control individual pixel exposure time that can enable
high dynamic range. Conventional approaches of using PCE camera involve
a time consuming and lossy process to reconstruct the original frames
and then use those frames for target tracking and classification. In this paper,
we present a deep learning approach that directly performs target tracking
and classification in the compressive measurement domain without any
frame reconstruction. Our approach has two parts: tracking and classification.
The tracking has been done using YOLO (You Only Look Once) and the
classification is achieved using Residual Network (ResNet). Extensive experiments
using mid-wave infrared (MWIR) and long-wave infrared (LWIR)
videos demonstrated the efficacy of our proposed approach.
References
[1]
Li, X., Kwan, C., Mei, G. and Li, B. (2006) A Generic Approach to Object Matching and Tracking. 3rd International Conference Image Analysis and Recognition, Lecture Notes in Computer Science, Póvoa de Varzim, 18-20 September 2006, 839-849.
https://doi.org/10.1007/11867586_76
[2]
Zhou, J. and Kwan, C. (2018) Tracking of Multiple Pixel Targets Using Multiple Cameras. 15th International Symposium on Neural Networks, Minsk, 25-28 June 2018, 484-493. https://doi.org/10.1007/978-3-319-92537-0_56
[3]
Zhou, J. and Kwan, C. (2018) Anomaly Detection in Low Quality Traffic Monitoring Videos Using Optical Flow. Pattern Recognition and Tracking, Vol. 10649.
[4]
Kwan, C., Zhou, J., Wang, Z. and Li, B. (2018) Efficient Anomaly Detection Algorithms for Summarizing Low Quality Videos. Pattern Recognition and Tracking, Vol. 10649. https://doi.org/10.1117/12.2303764
[5]
Kwan, C., Yin, J. and Zhou, J. (2018) The Development of a Video Browsing and Video Summary Review Tool. Pattern Recognition and Tracking, Vol. 10649.
https://doi.org/10.1117/12.2303654
[6]
Zhao, Z., Chen, H., Chen, G., Kwan, C. and Li, X.R. (2006) IMM-LMMSE Filtering Algorithm for Ballistic Target Tracking with Unknown Ballistic Coefficient. Proceedings SPIE, Vol. 6236, Signal and Data Processing of Small Targets.
https://doi.org/10.1117/12.665760
[7]
Zhao, Z., Chen, H., Chen, G., Kwan, C. and Li, X.R. (2006) Comparison of Several Ballistic Target Tracking Filters. Proceedings of American Control Conference, Minneapolis, 14-16 June 2006, 2197-2202.
[8]
Candes, E.J. and Wakin, M.B. (2008) An Introduction to Compressive Sampling. IEEE Signal Processing Magazine, 25, 21-30.
https://doi.org/10.1109/MSP.2007.914731
[9]
Kwan, C., Chou, B. and Kwan, L.M. (2018) A Comparative Study of Conventional and Deep Learning Target Tracking Algorithms for Low Quality Videos. 15th International Symposium on Neural Networks, Minsk, 25-28 June 2018, 521-531.
https://doi.org/10.1007/978-3-319-92537-0_60
[10]
Kwan, C., Chou, B., Yang, J. and Tran, T. (2019) Target Tracking and Classification Directly in Compressive Measurement for Low Quality Videos. Pattern Recognition and Tracking, Baltimore, 16-18 April 2019. https://doi.org/10.1117/12.2518496
[11]
Kwan, C., Chou, B., Echavarren, A., Budavari, B., Li, J. and Tran, T. (2018) Compressive Vehicle Tracking Using Deep Learning. IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York, 8-10 November 2018.
[12]
Zhang, J., Xiong, T., Tran, T., Chin, S. and Etienne-Cummings, R. (2016) Compact All-CMOS Spatio-Temporal Compressive Sensing Video Camera with Pixel-Wise Coded Exposure. Optics Express, 24, 9013-9024.
https://doi.org/10.1364/OE.24.009013
[13]
Tropp, J.A. (2004) Greed Is Good: Algorithmic Results for Sparse Approximation. IEEE Transactions on Information Theory, 50, 2231-2242.
https://doi.org/10.1109/TIT.2004.834793
[14]
Dao, M., Kwan, C., Koperski, K. and Marchisio, G. (2017) A Joint Sparsity Approach to Tunnel Activity Monitoring Using High Resolution Satellite Images. IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York, 19-21 October 2017, 322-328.
https://doi.org/10.1109/UEMCON.2017.8249061
[15]
Zhou, J., Ayhan, B., Kwan, C. and Tran, T. (2018) ATR Performance Improvement Using Images with Corrupted or Missing Pixels. Pattern Recognition and Tracking, Vol. 10649, 106490E.
[16]
Yang, J. and Zhang, Y. (2011) Alternating Direction Algorithms for 1-Problems in Compressive Sensing. SIAM Journal on Scientific Computing, 33, 250-278.
https://doi.org/10.1137/090777761
[17]
Applied Research LLC, Phase 1 Final Report, August 2016.
[18]
Yang, M.H., Zhang, K. and Zhang, L. (2012) Real-Time Compressive Tracking. European Conference on Computer Vision, Florence, 7-13 October 2012, 864-877.
[19]
Kwan, C., Chou, B., Yang, J., Rangamani, A., Tran, T., Zhang, J. and Etienne-Cummings, R. (2019) Target Tracking and Classification Directly Using Compressive Sensing Camera for SWIR Videos. Journal of Signal, Image, and Video Processing, 1-9. https://doi.org/10.1007/s11760-019-01506-4
[20]
Redmon, J. and Farhadi, A. (2018) YOLOv3: An Incremental Improvement.
[21]
He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/CVPR.2016.90
[22]
Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O. and Torr, P. (2016) Staple: Complementary Learners for Real-Time Tracking. Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 1401-1409.
https://doi.org/10.1109/CVPR.2016.156
[23]
Stauffer, C. and Grimson, W.E.L. (1999) Adaptive Background Mixture Models for Real-Time Tracking, Computer Vision and Pattern Recognition. IEEE Computer Society Conference, Vol. 2, Ft. Collins, 23-25 June 1999, 2246.
[24]
Ren S., He, K., Girshick, R. and Sun, J. (2015) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Advances in Neural Information Processing Systems, Montreal, 7-12 December 2015, 91-99.
