Musical Annotation and its Application to Music Retrieval and Summarization

Katsuhiko KAJI
Dept. of Computational Science and Engineering, Graduate School of Engineering, Nagoya University
Katashi NAGAO
Center for Information Media Studies, Nagoya University

1 Introduction

In recent years, research which append annotation data in semiautomatic on various digital contents, such as a document and video, is advanced from the necessity for annotation which adds metadata to contents.

By creating annotation data,the accuracy of contents reference is raised and conversion, automatic edit, etc. of contents are attained. Moreover, the useful knowledge over many people and the contents of those is sharable by exhibiting annotation.

However, research of creating annotation to the present work of art is not fully done.If annotation data can be given to a work of art, it can use for various services, such as description, education, amusement, and reference.

2 Musical Annotation

Especially in this research, we note music.Music is used in various places. It not only listen to, but also it is used in various scenes such as a lesson of karaoke, a piano classroom, and a school.In such a case, if there are useful annotation to each service, the contents of service can be enriched more.

Today,there are MusicXML and WEDELMUSIC which expresses music information by XML.It became possible to treat music easily by computer. Based on such XML, a score can be displayed or MIDI can be created.

However, all the information on music cannot be covered in such XML form. If MusicXML is taken for an example, although it is possible to describe information in detail to note each of bat information cannot be added to arbitrary parts, such as a title, a composer,expression marks, and lyrics.

Then, what type of information is short in music?If it is the position of the "author", by appending annotation which is the design for the detailed meaning of a title or the configuration which make the music easy to imagine.If it is a "service donor",he append annotation data which makes the service more rich in advance.In the case of a piano classroom,when student plays piano,the system can display attention point to play in real time.

In this research, We implemented the system to appending information for music as annotation and considered how musical annotation could apply by using MusicXML as a base.

3 Implementation of MiXA(MusicXML Annotator)

We implemented the system named MiXA(MusicXML Annotator).It can append annotation datato arbitrary part of musical elements.

no caption

Fugure1: no caption

The outline of a system is as follows.

MusicXML is saved beforehand at Xindice which is an XML database. Servlet is moving to the server where MusicXML is saved.

User accesses specific URL from Internet Explorer.Then, Servlet takes out DOM (Document Object Model) of MusicXML from Xindice.Furthermore, MusicXML is changed into SVG (Scalable Vector Graphics) of a score, and it displays on a browser.XPath is written that it turns out which part of the original MusicXML in each of SVG objects.User chooses SVG objects, such as a note, a title, and an expression mark, with a mouse, and describes the annotation data of the part.Annotation data is not directly added to the original MusicXML, but is saved as another XML to Xindice.If annotation data is directry added to MusicXML, the specification of MusicXML will be changed and it will become impossible to use with other applications.

Annotation type which a user can give is as follows.

  1. comment:Comment and interpretation

  2. performance:The point which it is careful of by performance

  3. chords:Chord advance

  4. skeleton:Musical rough skeleton

  5. tablature:Performance method

  6. something:Free annotation

"something" was prepared the sake that user is going to use new annotation with original application. When there isn't relevant annotation type, user can select "something".

4 Retrieval and Summarization by using MiXA

By using annotation data created by MusicXML and MiXA, we implemented prototype system which sums music and creates playlist.

4.1 Musical summation tool

We made the prototype system which can summarize music freely using "skeleton".By choosing the parts which user wanted to listen to or sing, user can get new MIDI which consisted of only selected parts from MIDI created by the original MusicXML.

4.2 Playlist creator

User can retrieve music including the annotation data.We made the play list creation tool which creates the MIDI list based on this reference result.For example, the music by which "Christmas" is contained in annotatin data with a title, lyrics and a composer is outputted to the reference result in the keyword "Christmas."Then the music play list related to "Christmas" can be obtained.Moreover, combining with summation tool, user can get a playlist which has only intros or choruses.

5 Summary

This paper described the necessity for sufficient annotation in music, and implemented MiXA which can append musical annotation data.Moreover, we implemented the prototype system which sums music and creates playlist.

A future subject is listed to below.

If the structure of code advance and tablature can be described in more detail, user can retrieve more flexibly.Like present system, all annotation data is appended as string.So we will be able to append links, tree structure and specific tablature.

It is hard to say that all the information on music can be covered only for the information given this time.For example, vibrato is one of the most important elements in popular ballad.Appropriate information is also required by the musical kind.Furthermore, it is due to use for various tasks, such as quotation, complement, and association.