In this paper, we present machine learning algorithms and systems for similar video retrieval. Here, the query is itself a video. For the similarity measurement, exemplars, or representative frames in each video, are extracted by unsupervised learning. For this learning, we chose the order-aware competitive learning. After obtaining a set of exemplars for each video, the similarity is computed. Because the numbers and positions of the exemplars are different in each video, we use a similarity computing method called M-distance, which generalizes existing global and local alignment methods using followers to the exemplars. To represent each frame in the video, this paper emphasizes the Frame Signature of the ISO/IEC standard so that the total system, along with its graphical user interface, becomes practical. Experiments on the detection of inserted plagiaristic scenes showed excellent precision-recall curves, with precision values very close to 1. Thus, the proposed system can work as a plagiarism detector for videos. In addition, this method can be regarded as the structuring of unstructured data via numerical labeling by exemplars. Finally, further sophistication of this labeling is discussed.
References
[1]
Hu, W., Xie, N., Li, L., Zeng, X. and Maybank, S. (2011) A Survey on Visual Content-Based Video Indexing and Retrieval. IEEE Transactions on Systems, Man, and Cybernetic, 41, 797-819. https://doi.org/10.1109/TSMCC.2011.2109710
[2]
ISO/IEC 15938-3:2002/Amd.4:2010, Information Technology—Multimedia Content Description Interface—Part 3: Visual, AMENDMENT 4: Video Signature Tools.
[3]
Levenshtein, V.I. (1966) Binary Codes of Correcting Deletions, Insertions, and Reversals. Soviet Physics Doklady, 10, 707-710.
[4]
Sakoe, H. and Chiba, S. (1978) Dynamic Programming Algorithm Optimization for Spoken Word Recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing, 26, 43-49. https://doi.org/10.1109/TASSP.1978.1163055
[5]
Needleman, S.B. and Wunsch, C.D. (1970) A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins. Journal of Molecular Biology, 48, 443-453. https://doi.org/10.1016/0022-2836(70)90057-4
[6]
Smith, T.F. and Waterman, M.S. (1981) Identification of Common Molecular Subsequences. Journal of Molecular Biology, 147, 195-197.
https://doi.org/10.1016/0022-2836(81)90087-5
[7]
Messing, D.S., van Beek, P. and Errico, J.H. (2001) The MPEG-7 Colour Structure Descriptor: Image Description Using Colour and Local Spatial Information. Proceedings 2001 International Conference on Image Processing, Thessaloniki, Greece, 1, 670-673. https://doi.org/10.1109/ICIP.2001.959134
[8]
Paschalakis, S., Iwamoto, K., Brasnett, P., Sprlian, N., Oami, R., Nomura, T., Yamada, A. and Bober, M. (2012) The MPEG-7 Video Signature Tools for Content Identification. IEEE Transactions on Circuits and Systems for Video Technology, 22, 1050-1063. https://doi.org/10.1109/TCSVT.2012.2189791
[9]
ISO/IEC 15938-3:2002, Information Technology—Multimedia Content Description Interface—Part 3: Visual.
[10]
Horie, T., Shikano, A., Iwase, H. and Matsuyama, Y. (2015) Learning Algorithms and Frame Signatures for Video Similarity Ranking. Lecture Notes in Computer Science, 9489, 147-157. https://doi.org/10.1007/978-3-319-26532-2_17
[11]
Frey, B.J. and Dueck, D. (2007) Clustering by Passing Messages between Data Points. Science, 315, 972-976. https://doi.org/10.1126/science.1136800
[12]
Horie, T., Moriwaki, M, Yokote, R., Ninomiya, S., Shikano, A. and Matsuyama, Y. (2014) Similar-Video Retrieval via Learned Exemplars and Time-Warped Alignment. Lecture Notes in Computer Science, 8836, 85-94.
https://doi.org/10.1007/978-3-319-12643-2_11
[13]
Matsuyama, Y. (1996) Harmonic Competition: A Self-Organizing Multiple Criteria Optimization. IEEE Transactions on Neural Networks, 7, 652-668.
https://doi.org/10.1109/72.501723
[14]
Matsuyama, Y., Shikano, A., Iwase, H. and Horie, T. (2015) Order-Aware Exemplars for Structuring Video Sets: Clustering, Aligned Matching and Retrieval by Similarity. Proceedings of the International Joint Conference on Neural Networks, Killarney, Ireland, 1-10. https://doi.org/10.1109/IJCNN.2015.7280423
[15]
Equitz, W.H. (1989) A New Vector Quantization Clustering Algorithm. IEEE Transactions on Acoustics, Speech, and Signal Processing, 37, 1568-1575.
https://doi.org/10.1109/29.35395
[16]
Ward, J.H. (1963) Hierarchical Grouping to Optimize an Objective Function. Journal of the American Statistical Association, 58, 236-244.
https://doi.org/10.1080/01621459.1963.10500845
Thomee, B., Shamma, D.A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D. and Li, L. (2016) YFCC100M: The New Data in Multimedia Research. Communications of the ACM, 59, 64-73. https://doi.org/10.1145/2812802
[19]
Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014) Generative Adversarial Nets. Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, Canada, 2, 2672-2680.
