Annotation-Based Multimedia Summarization and Translation

Katashi NAGAO
名古屋大学 大学院 情報科学研究科
Shigeki OHIRA
名古屋大学 情報基盤センター
Mitsuhiro YONEOKA


This paper presents techniques for multimedia annotation and theirapplication to video summarization and translation.Our tool for annotation allows users to easily create annotation including voice transcripts, video scene descriptions,and visual/auditory object descriptions.The module for voice transcription is capable of multilingual spoken language identification and recognition.A video scene description consists of semi-automatically detected keyframes of each scene in a video clip and time codes of scenes.A visual object description is created by tracking and interactivenaming of people and objects in video scenes.The text data in the multimedia annotation are syntactically andsemantically structured using linguistic annotation.The proposed multimedia summarization works upon a multimodal documentthat consists of a video, keyframes of scenes, and transcripts ofthe scenes.The multimedia translation automatically generates several versions ofmultimedia content in different languages.