Music retrieval

Abstract. Recent research has shown the Linked Data cloud to be a potentially ideal basis for improving user experience when interacting with Web content across different applications and domains. Using the explicit knowledge of datasets, however, is neither sufficient nor straightforward. Dataset knowledge is often not uniformly organized, thus it is generally unknown how to query for it. To deal with these issues, we propose a dataset analysis approach based on knowledge patterns, and show how the recognition of ...

Chroma-based audio features are a well-established tool for analyzing and comparing music data. By identifying spectral components that differ by a musical octave, chroma features show a high degree of invariance to variations in timbre. In this paper, we describe a novel procedure for making chroma features even more robust to changes in timbre and instrumentation while keeping their discriminative power. Our idea is based on the generally accepted observation that the lower mel-frequency cepstral coefficients (MFCCs) are closely related to timbre. Now, instead of keeping the lower coefficients, we will discard them and only keep the upper coefficients. Furthermore, using a pitch scale instead of a mel scale allows us to project the remaining coefficients onto the twelve chroma bins. Our systematic experiments show that the resulting chroma features have indeed gained a significant boost towards timbre invariance.

The intentionally ambiguous expression "Popular Music Browser" reflects the two main goals of this project, which started in 1998, at Sony CSL laboratory. First, we are interested in human-centered issues related to browsing "Popular Music". Popular here means that the music accessed to is widely distributed, and known to many listeners. Second, we consider "popular browsing" of music, i.e. making music accessible to non specialists (music lovers), and allowing sharing of musical tastes and information within communities, departing from the usual, single user view of digital libraries. This research project covers all areas of the music-to-listener chain, from music descriptiondescriptor extraction from the music signal, or data mining techniques -, similarity based access and novel music retrieval methods such as automatic sequence generation, and user interface issues. This paper describes the scientific and technical issues at stake, and the results obtained, and is illustrated by prototypes developed within the European IST project Cuidado.

To facilitate information retrieval of large-scale music databases, the detection of musical concepts, or auto-tagging, has been an active research topic. This paper concerns the use of concept correlations to improve musical concept detection. We propose to formulate concept detection as an ordinal regression problem to explicitly take advantage of the ordinal relationship between concepts and avoid the data imbalance problem of conventional multi-label classification methods. To further improve the detection accuracy, we propose to leverage the co-occurrence patterns of concepts for context fusion and employ concept selection to remove irrelevant or noisy concepts. Evaluation on the cal500 dataset shows that we are able to improve the detection accuracy of 174 concepts from 0.2513 to 0.2924.

In this paper we present the Audio Effects Ontology for the ontological representation of audio effects in music production workflows. Designed as an extension to the Studio Ontology, its aim is to provide a framework for the detailed description and sharing of information about audio effects, their implementations, and how they are applied in realworld production scenarios. The ontology enables capturing and structuring data about the use of audio effects and thus facilitates reproducibility of audio effect application, as well as the detailed analysis of music production practices. Furthermore, the ontology may inform the creation of metadata standards for adaptive audio effects that map high-level semantic descriptors to control parameter values. The ontology is using Semantic Web technologies that enable knowledge representation and sharing, and is based on modular ontology design methodologies. It is evaluated by examining how it fulfils requirements in a number of production and retrieval use cases.

We present a measure of the similarity of the long-term structure of musical pieces. The system deals with raw polyphonic data. Through unsupervised learning, we generate an abstract representation of music -the "texture score". This "texture score" can be matched to other similar scores using a generalized edit distance, in order to assess structural similarity. We notably apply this algorithm to the retrieval of different interpretations of the same song within a music database.

We consider two formulations of the computational problem of transposition-invariant, time-offset tolerant, meterinvariant, and timescale invariant polyphonic music retrieval. We provide algorithms for both that are scalable in the sense that space requirements are asymptotically linear and queries are efficient for large databases of music. The focus is on cases where a query pattern M consisting of m events is to be matched against a database N consisting of n events, and m ≪ n. The database is assumed to be polyphonic, and the algorithms support polyphonic queries. We are interested in finding exact and proximate occurrences of the query pattern. The first problem considered is that of finding the minimum directed Hausdorff distance from M to N. We give a (2 + ǫ)-approximation algorithm that solves this problem in O (nm) query time and O (n) space. The second problem is that of finding all maximal subset matches of M in N , and we give an algorithm that solves this problem in O m 3 (k + 1) query time and O w 2 n space, where w represents the maximum window size and k is the number of matches. Using the same method, the problem can be solved in O (m (k + 1)) query time and O (wn) space if we do not require the timescale invariance property. The latter query time is asymptotically optimal for the given problem.

Emotion based music retrieval is a music player based on user's emotion. Many music devices and mobile music players are used to listen to music. A practical problem is selection of desired music. Nowadays many devices are integrated with cameras. This paper gives how to take advantage of these one camera systems. In this proposed system emotion is derived automatically from hand gestures where hand gestures are captured by camera. The data recorded from hand gestures are coupled with musical emotion. Users can search through music collection based on emotional character of music which expressed from hand gestures. The main aim of this project is to design emotion based user interface for music retrieval. The algorithms were implemented in the system using C++. The system uses the Intel OpenCV library for image processing. This paper is implemented by using ARM9 micro controller. The camera is connected to the controller through USB device. The microcontroller is connected to PC through serial communication. The controller processes the information and monitors the results as controlled songs on PC.

Symmetry turns out to be a very important concept in the analysis of music perception and its relationship to speech perception. There are five or maybe six identifiable symmetries of music perception. These are invariances under six corresponding types of transformation: pitch translation, octave translation, time translation, time scaling, amplitude scaling and (possibly) pitch reflection. Different symmetries apply to different aspects of music.

Musical scores are traditionally retrieved by title, composer or subject classification. Just as multimedia computer systems increase the range of opportunities available for presenting musical information, so they also offer new ways of posing musically-oriented queries. This paper shows how scores can be retrieved from a database on the basis of a few notes sung or hummed into a microphone. The design of such a facility raises several interesting issues pertaining to music retrieval. We first describe an interface that transcribes acoustic input into standard music notation. We then analyze string matching requirements for ranked retrieval of music and present the results of an experiment which tests how accurately people sing well known melodies. The performance of several string matching criteria are analyzed using two folk song databases. Finally, we describe a prototype system which has been developed for retrieval of tunes from acoustic input and evaluate its performance.

Human listeners are able to recognize structure in music through the perception of repetition and other relationships within a piece of music. This work aims to automate the task of music analysis. Music is "explained" in terms of embedded relationships, especially repetition of segments or phrases. The steps in this process are the transcription of audio into a representation with a similarity or distance metric, the search for similar segments, forming clusters of similar segments, and explaining music in terms of these clusters. Several transcription methods are considered: monophonic pitch estimation, chroma (spectral) representation, and polyphonic transcription followed by harmonic analysis. Also, several algorithms that search for similar segments are described. These techniques can be used to perform an analysis of musical structure, as illustrated by examples.

To achieve a good balance between matching accuracy and computation efficiency is a key challenge for Query-by-Humming (QBH) system. In this paper, we propose an approach of n-gram based fast match. Our n-gram method uses a robust statistical note transcription as well as error compensation method based on the analysis of frequent transcription errors. The effectiveness of our approach has been evaluated on a relatively large melody database with 5223 melodies. The experimental results show that when the searching space was reduced to only 10% of the whole size, 90% of the target melodies were preserved in the candidates, and 88% of the match accuracy of system was kept. Meanwhile, no obvious additional computation was applied.

This paper extends the familiar "query by humming" music retrieval framework into the polyphonic realm. As humming in multiple voices is quite difficult, the task is more accurately described as "query by audio example", onto a collection of scores. To our knowledge, we are the first to use polyphonic audio queries to retrieve from polyphonic symbolic collections. Furthermore, as our results will show, we will not only use an audio query to retrieve a known-item symbolic piece, but we will use it to retrieve an entire set of real-world composed variations on that piece, also in the symbolic format. The harmonic modeling approach which forms the basis of this work is a new and valuable technique which has both wide applicability and future potential.V

This paper describes a distributed collaborative wiki-based platform that has been designed to facilitate the development of Semantic Web applications. The applications designed using this platform are able to build semantic data through the cooperation of different developers and to exploit that semantic data. The paper shows a practical case study on the application VPOET, and how an application based on Google Gadgets has been designed to test VPOET and let human users exploit the semantic data created. This practical example can be used to show how different Semantic Web technologies can be integrated into a particular Web application, and how the knowledge can be cooperatively improved.

This paper explores the extraction of melodic pitch contour from the polyphonic soundtrack of a song. The motivation for this work lies in developing automatic tools for the melodybased indexing of the database in a music retrieval system. The melody is assumed to be carried by the singer's voice accompanied mainly by percussive instruments. This scenario is typical of a large class of Indian movie songs. The challenges raised by this application are presented. A pitch detection method based on a perceptual model is shown to be a promising approach to the tracking of voice pitch in the presence of strong percussive background.

Music information retrieval is a field of rapidly growing commercial interest. This paper describes TANSEN, a query-by-humming based music retrieval system under development at IIT, Bombay. Named after the legendary musician (and a tenuous acronym for "TA-Note Song Extractor-Navigator), the system is designed to accept acoustic queries in the form of sung fragments, to search a database of Indian film songs. Algorithms for the extraction of melody from the query signal, and pattern matching for search and retrieval from the database are presented. The user interface is described, and experimental results obtained on a prototype version are reported.

We have explored methods for music information retrieval for polyphonic music stored in the MIDI format. These methods use a query, expressed as a series of notes that are intended to represent a melody or theme, to identify similar pieces. Our work has shown that a three-phase architecture is appropriate for this task, in which the first phase is melody extraction, the second is standardisation, and the third is query-to-melody matching. We have investigated and systematically compared algorithms for each of these phases. To ensure that our results are robust, we have applied methodologies that are derived from text information retrieval: we developed test collections and compared different ways of acquiring test queries and relevance judgements. In this paper we review this program of work, compare to other approaches to music information retrieval, and identify outstanding issues.

Most research into music information retrieval thus far has only examined music from the western tradition. However, music of other origins often conforms to different tuning systems. Therefore there are problems both in representing this music as well as finding matches to queries from these diverse tuning systems. We discuss the issues associated with microtonal music retrieval and present some preliminary results from an experiment in applying scoring matrices to microtonal matching.

Accurately finding audio recordings in response to symbolic queries is one of the key challenges in the field of music information retrieval. Pitch is one of the main features of music; in this paper we propose and evaluate approaches for using pitch information in polyphonic symbolic queries to retrieve full tracks of audio recordings. The audio data is first converted into symbolic data, using an automated transcription process. This is a noisy process, adding up to three times as many notes to the transcription than are actually present. Nevertheless, recordings can be accurately retrieved by manually-constructed queries (either in full or truncated) using the longest common subsequence algorithm (and a sliding window if the queries are truncated). Precision at 1 of about 80% was achieved, and around 85% of queries return correct answers in the top 10 from a collection of 1808 recordings. Truncated queries are as effective as untruncated queries for retrieving correct answers in the first rank position. Thus, the burden on users is reduced as they only need to produce a small fraction of a song as a query.

Managing and analyzing music scores content is a known issue for digital libraries. Despite their inarguable complexity, these contents are often ignored by digital libraries, where music scores are mostly treated as simple digital images. This paper reports on the development and application of MusicOWL, an ontology to formally describe music scores contents for western music. Unlike other e orts on the subject, MusicOWL provides a comprehensive vocabulary for annotating music scores, which covers important music-related information, such as melodies, dynamics, and tonalities. We also report on the Linked Music Score dataset creation, including all necessary steps from scanned music scores to RDF triples, and how this linked music score data enables users to search for music scores by using handy queries, thus translating the needs of many music professionals, data curators and online learners. Additionally, we introduce and analyze the search engine WWU Music Score Portal, which uses the approach proposed in this work to improve music scores discovery and support online learning of music.

Jazz is a musical tradition that is just over 100 years old; unlike in other Western musical traditions, improvisation plays a central role in jazz. Modelling the domain of jazz poses some ontological challenges due to specificities in musical content and performance practice, such as band lineup fluidity and importance of short melodic patterns for improvisation. This paper presents the Jazz Ontology-a semantic model that addresses these challenges. Additionally, the model also describes workflows for annotating recordings with melody transcriptions and for pattern search. The Jazz Ontology incorporates existing standards and ontologies such as FRBR and the Music Ontology. The ontology has been assessed by examining how well it supports describing and merging existing datasets and whether it facilitates novel discoveries in a music browsing application. The utility of the ontology is also demonstrated in a novel framework for managing jazz related music information. This involves the population of the Jazz Ontology with the metadata from large scale audio and bibliographic corpora (the Jazz Encyclopedia and the Jazz Discography). The resulting RDF datasets were merged and linked to existing Linked Open Data resources. These datasets are publicly available and are driving an online application that is being used by jazz researchers and music lovers for the systematic study of jazz.

Recognizing a musical excerpt without necessarily retrieving its title typically reflects the existence of a memory system dedicated to the retrieval of musical knowledge. The functional distinction between musical and verbal semantic memory has seldom been investigated. In this fMRI study, we directly compared the musical and verbal memory of 20 nonmusicians, using a congruence task involving automatic semantic retrieval and a familiarity task requiring more thorough semantic retrieval. In the former, participants had to access their semantic store to retrieve musical or verbal representations of melodies or expressions they heard, in order to decide whether these were then given the right ending or not. In the latter, they had to judge the level of familiarity of musical excerpts and expressions. Both tasks revealed activation of the left inferior frontal and posterior middle temporal cortices, suggesting that executive and selection processes are common to both verbal and musical retrieval. Distinct patterns of activation were observed within the left temporal cortex, with musical material mainly activating the superior temporal gyrus and verbal material the middle and inferior gyri. This cortical organization of musical and verbal semantic representations could explain clinical dissociations featuring selective disturbances for musical or verbal material.

We study the problem of identifying repetitions under transposition and time-warp invariances in polyphonic symbolic music. Using a novel onset-time-pair representation, we reduce the repeating pattern discovery problem to instances of the classical problem of finding the longest increasing subsequences. The resulting algorithm works in O(n 2 log n) time where n is the number of notes in a musical work. We also study windowed variants of the problem where onset-time differences between notes are restricted, and show that they can also be solved in O(n 2 log n) time using the algorithm.

The modeling of music as a language is a core issue for a wide range of applications such as polyphonic music retrieval, automatic style identification, audio to symbolic music transcription and computer-assisted composition. In this paper, we focus on the modeling of chord sequences by probabilistic N-grams. Previous studies using these models have achieved limited success, due to overfitting and to the use of a single chord labeling scheme. We investigate these issues using model smoothing and selection techniques initially designed for spoken language modeling. This approach is evaluated over a set of songs by The Beatles, considering several chord labeling schemes. Initial results show that the accuracy of N-grams is increased but that additional improvements may still be achieved in the future using more advanced, possibly music-specific, smoothing techniques.

The Linked Data paradigm has been used to publish a large number of musical datasets and ontologies on the Semantic Web, such as MusicBrainz, AcousticBrainz, and the Music Ontology. Recently, the MIDI Linked Data Cloud has been added to these datasets, representing more than 300,000 pieces in MIDI format as Linked Data, opening up the possibility for linking fine-grained symbolic music representations to existing music metadata databases. Despite the dataset making MIDI resources available in Web data standard formats such as RDF and SPARQL, the important issue of finding meaningful links between these MIDI resources and relevant contextual metadata in other datasets remains. A fundamental barrier for the provision and generation of such links is the difficulty that users have at adding new MIDI performance data and metadata to the platform. In this paper, we propose the Semantic Web MIDI Tape, a set of tools and associated interface for interacting with the MIDI Linked Data Cloud by enabling users to record, enrich, and retrieve MIDI performance data and related metadata in native Web data standards. The goal of such interactions is to find meaningful links between published MIDI resources and their relevant contextual metadata. We evaluate the Semantic Web MIDI Tape in various use cases involving user-contributed content, MIDI similarity querying, and entity recognition methods, and discuss their potential for finding links between MIDI resources and metadata.

This paper extends the familiar "query by humming" music retrieval framework into the polyphonic realm. As humming in multiple voices is quite difficult, the task is more accurately described as "query by audio example", onto a collection of scores. To our knowledge, we are the first to use polyphonic audio queries to retrieve from polyphonic symbolic collections. Furthermore, as our results will show, we will not only use an audio query to retrieve a known-item symbolic piece, but we will use it to retrieve an entire set of real-world composed variations on that piece, also in the symbolic format. The harmonic modeling approach which forms the basis of this work is a new and valuable technique which has both wide applicability and future potential.V

In this paper, we present a novel approach for music summarization based on music structure analysis. From the audio signal, we first extract the note onset representing the time tempo of the song and the music structure analysis can be performed based on this tempo information. After music content has been structured into different semantic regions such as Introduction (Intro), Verse, Chorus, Ending (Outro), etc., the final music summary can be created with chorus and music phrases which are included anterior or posterior to selected chorus to get the desired length of the final summary. In this way, we can guarantee that the summaries begin and end at meaningful music phrase boundaries, which is a difficult problem for existing music summarization methods. Experiments show our proposed method can capture the main theme of the music compared to the ideal summaries selected by music experts and user subjective evaluation indicates our proposed method has a good performance.

A n immeasurable amount of multimedia information is available today-in digital archives, on the Web, in broadcast data streams, and in personal and professional databases-and this amount continues to grow. Yet, the value of that information depends on how easily we can manage, find, retrieve, access, and filter it. The transition between two millennia abounds with new ways to produce, offer, filter, search, and manage digitized multimedia information.

Employing existing ontologies including GeoNames and the Music Ontology, we present a proof-of-concept system that demonstrates the utility of Linked Data for enhancing the application of MIR workflows, both when curating collections of signal for analysis, and publishing results that can be easily correlated to these, and other, collection sets. By way of example we gather and publish metadata describing signal collections derived from the country of an artist; genre analysis over these collections and integration of collection and ...

We describe a method that aligns polyphonic audio recordings of music to symbolic score information in standard MIDI files without the difficult process of polyphonic transcription. By using this method, we can search through a MIDI database to find the MIDI file corresponding to a polyphonic audio recording.

We apply a new machine learning tool, kernel combination, to the task of semantic music retrieval. We use 4 different types of acoustic content and social context feature sets to describe a large music corpus and derive 4 individual kernel matrices from these feature sets. Each kernel is used to train a support vector machine (SVM) classifier for each semantic tag (e.g., 'aggressive', 'classic rock', 'distorted electric guitar') in a large tag vocabulary. We examine the individual performance of each feature kernel and then show how to learn an optimal linear combination of these kernels using convex optimization. We find that the retrieval performance of the SVMs trained using the combined kernel is superior to SVMs trained using the best individual kernel for a large number of tags. In addition, the weights placed on individual kernels in the linear combination reflect the relative importance of each feature set when predicting a tag.

When attempting to annotate music, it is important to consider both acoustic content and social context. This paper explores techniques for collecting and combining multiple sources of such information for the purpose of building a query-by-text music retrieval system. We consider two representations of the acoustic content (related to timbre and harmony) and two social sources (social tags and web documents). We then compare three algorithms that combine these information sources: calibrated score averaging (CSA), RankBoost, and kernel combination support vector machines (KC-SVM). We demonstrate empirically that each of these algorithms is superior to algorithms that use individual information sources.

TagClouds is a popular visualization for the collaborative tags. However it has some instinct problems such as linguistic issues, high semantic density and poor understanding of hierarchical structure and semantic relation between tags. In this paper we investigate the ways to support semantic understanding of collaborative tags and propose an improved visualization named TagClusters. Based on the semantic analysis of the collaborative tags in Last.fm, the semantic similar tags are clustered into different groups and the visual distance represents the semantic similarity between tags, and thus the visualization offers a better semantic understanding of collaborative tags. A comparative evaluation is conducted with TagClouds and TagClusters based on the same tags collection. The results indicate that TagClusters has advantages in supporting efficient browsing, searching, impression formation and matching. In the future work, we will explore the possibilities of supporting tag recommendation and tag-based Music Retrieval based on TagClusters.

Existing literature has discussed the use of rule-based systems for intelligent mixing. These rules can either be explicitly defined by experts, learned from existing datasets, or a mixture of both. For such mixing rules to be transferable between different systems and shared online, we propose a representation using the Rule Interchange Format (RIF) commonly used on the Semantic Web. Systems with differing capabilities can use OWL reasoning on those mixing rule sets to determine subsets which they can handle appropriately. We demonstrate this by means of an example web-based tool which uses a logical constraint solver to apply the rules in real time to sets of audio tracks annotated with features.

This paper introduces the Audio Effect Ontology (AUFX-O) building on previous theoretical models describing audio processing units and workflows in the context of music production. We discuss important conceptualisations of different abstraction layers, their necessity to successfully model audio effects, and their application method. We present use cases concerning the use of effects in music production projects and the creation of audio effect metadata facilitating a linked data service exposing information about effect implementations. By doing so, we show how our model facilitates knowledge sharing, reproducibility and analysis of audio production workflows.

Feature extraction algorithms in Music Informatics aim at deriving statistical and semantic information directly from audio signals. These may be ranging from energies in several frequency bands to musical information such as key, chords or rhythm. There is an increasing diversity and complexity of features and algorithms in this domain and applications call for a common structured representation to facilitate interoperability, reproducibility and machine interpretability. We propose a solution relying on Semantic Web technologies that is designed to serve a dual purpose (1) to represent computational workflows of audio features and (2) to provide a common structure for feature data to enable the use of Open Linked Data principles and technologies in Music Informatics. The Audio Feature Ontology is based on the analysis of existing tools and music informatics literature, which was instrumental in guiding the ontology engineering process. The ontology provides a descriptive framework for expressing different conceptualisations of the audio feature extraction domain and enables designing linked data formats for representing feature data. In this paper, we discuss important modelling decisions and introduce a harmonised ontology library consisting of modular interlinked ontologies that describe the different entities and activities involved in music creation, production and publishing.

University Carlos III of Madrid Department of Computer Science Leganés, Madrid, Spain {jurbano, jmorato, mmarrero, dmandres}@inf.uc3m.es ... ABSTRACT Music similarity tasks, where musical pieces similar to a query should be retrieved, are quite troublesome to evaluate. ...

In this work, a novel representation system for symbolic music is described. The proposed representation system is graph-based and could theoretically represent music both from a horizontal (contrapuntal) and from a vertical (harmonic) point of view, by keeping into account contextual and harmonic information. It could also include relationships between internal variations of motifs and themes. This is achieved by gradually simplifying the melodies and generating layers of reductions that include only the most important notes from a structural and harmonic viewpoint. This representation system has been tested in a music information retrieval task, namely melodic similarity, and compared to another system that performs the same task but does not consider any contextual or harmonic information, showing how the structural information is needed in order to find certain relations between musical pieces. Moreover, a new dataset consisting of more than 5000 leadsheets is presented, with additional meta-musical information taken from different web databases, including author, year of first performance, lyrics, genre and stylistic tags. CCS CONCEPTS • Information systems → Music retrieval; • Applied computing → Sound and music computing;

This paper extends the familiar "query by humming" music retrieval framework into the polyphonic realm. As humming in multiple voices is quite difficult, the task is more accurately described as "query by audio example", onto a collection of scores. To our knowledge, we are the first to use polyphonic audio queries to retrieve from polyphonic symbolic collections. Furthermore, as our results will show, we will not only use an audio query to retrieve a known-item symbolic piece, but we will use it to retrieve an entire set of real-world composed variations on that piece, also in the symbolic format. The harmonic modeling approach which forms the basis of this work is a new and valuable technique which has both wide applicability and future potential.V

Sonic Visualiser is the name for an implementation of a system to assist study and comprehension of the contents of audio data, particularly of musical recordings. It is a C++ application with a Qt4 GUI that runs on Windows, Mac, and Linux. It embodies a number of concepts which are intended to improve interaction with audio data and features, most notably with respect to the representation of time-synchronous information. The architecture of the application allows for easy integration of third party algorithms for the extraction of low and mid-level features from musical audio data. This paper describes some basic principles and functionalities of Sonic Visualiser.

This paper extends the familiar "query by humming" music retrieval framework into the polyphonic realm. As humming in multiple voices is quite difficult, the task is more accurately described as "query by audio example", onto a collection of scores. To our knowledge, we are the first to use polyphonic audio queries to retrieve from polyphonic symbolic collections. Furthermore, as our results will show, we will not only use an audio query to retrieve a known-item symbolic piece, but we will use it to retrieve an entire set of real-world composed variations on that piece, also in the symbolic format. The harmonic modeling approach which forms the basis of this work is a new and valuable technique which has both wide applicability and future potential.V

One of the key missions of the Polifonia project is to build and interlink knowledge graphs of European musical cultural heritage. These interconnected music knowledge graphs act as a data backbone for the project. This deliverable is centred around one of the ten Polifonia pilots: INTERLINK. The INTERLINK pilot is not bound to a specific institutional dataset, collection or use case; but rather summarises the common needs of all pilots of representing their data as music knowledge graphs, and finding meaningful connections among them. Existing work in the relevant areas of knowledge engineering, knowledge graph construction, and knowledge graph interlinking is promising but presents a number of pitfalls in its direct application to Polifonia from the point of view of achieving this knowledge graph building and interlinking: (a) existing knowledge engineering methodologies do not account for "in-between" knowledge graph construction processes that are not fully distributed nor local; (b) in an integrated vision of European musical cultural heritage through knowledge graphs, both musical content and its 'context' or metadata need to co-exist; and (c) no current techniques address the issue of finding missing links between different music knowledge graphs at scale, accounting for this coexistence of musical content and context in the same knowledge graph. This deliverable describes our efforts, particularly gathered in Polifonia's WP2 and Task T2.3, in addressing these challenges, delivering: (a) a scalable methodology for knowledge graph construction that is well suited for the project and sits in-between the two classic extremes of ontology engineering; (b) a music knowledge graph construction framework that leverages modules of the Polifonia Ontology Network and the SPARQL Anything tool; and (c) a music knowledge graph interlinking framework, based on both heuristics and neural network-based link prediction through music knowledge graph embeddings. The latter two are evaluated in two different use-cases, the ChoCo-KG chord annotation knowledge graph, and the MIDI2vec music knowledge graph interlinking method, with encouraging results.

Integration between different data formats, and between data belonging to different collections, is an ongoing challenge in the MIR field. Semantic Web tools have proved to be promising resources for making different types of music information interoperable. However, the use of these technologies has so far been limited and scattered in the field. To address this, the Polifonia project 1 is developing an ontological ecosystem that can cover a wide variety of musical aspects (musical features, instruments, emotions, performances). In this paper, we present the Polifonia Ontology Network, an ecosystem that enables and fosters the transition towards Semantic MIR.

Existing literature has discussed the use of rule-based systems for intelligent mixing. These rules can either be explicitly defined by experts, learned from existing datasets, or a mixture of both. For such mixing rules to be transferable between different systems and shared online, we propose a representation using the Rule Interchange Format (RIF) commonly used on the Semantic Web. Systems with differing capabilities can use OWL reasoning on those mixing rule sets to determine subsets which they can handle appropriately. We demonstrate this by means of an example web-based tool which uses a logical constraint solver to apply the rules in real time to sets of audio tracks annotated with features.

Given two n-bit (cyclic) binary strings, A and B, represented on a circle (necklace instances), let each sequence have the same number (k) of 1's. We are interested in computing the cyclic swap distance between A and B, i.e. the minimum number of swaps needed to convert A to B, minimized over all possible rotations of B. We show that, given the compressed representation of A and B, this distance may be computed in O(k2).

The problem of music retrieval by sung query consists of building a machine capable of simulating the cognitive process of identifying a musical piece from a few sung notes of its melody. In this paper, the algorithms of pitch tracking, onset detection and melody matching used in the system Tararira are described. Much effort has been put on automatic transcription of singing voice as it is a key factor in the overall performance. A novel way of combining note by note matching with a recent approach based on pitch time series matching is introduced. Resumen. El problema de búsqueda de música por tarareo consiste en construir un sistema capaz de simular el proceso cognitivo de identificar una pieza musical a partir de unas pocas notas cantadas de su melodía. En este artículo se describen los algoritmos de detección de altura, segmentación de audio en notas y comparación de melodías utilizados en el sistema Tararira. Se concentran esfuerzos en la transcripción automática de la voz cantada ya que es determinante en el desempeño del sistema. Para la comparación de melodías se propone una forma de combinar los enfoques basados en notas y series temporales, considerados antagónicos hasta el momento.

New interactive music services have emerged, despite currently using proprietary file formats. Having a standardized file format could benefit the interoperability between these services. In this regard, the ISO/IEC Moving Picture Experts Group (MPEG) issued a new standard, the so called, MPEG-A: Interactive Music Application Format (IM AF). The purpose of this paper is to describe the design and implementation of an IM AF codec and its integration into Sonic Visualiser. In this way, the visualization of the chords or the pitch of the main melody aligned in time with the song's lyrics is achieved. Furthermore, this integration provides the semantic audio research community with a test-bed for further development and comparison of new Sonic Visualiser VAMP plug-ins, e.g., for the conversion of singing voice to text and/or automatic highlighting of lyrics for karaoke applications.