MUMIS - A Multimedia Indexing and Searching Environment

1999

We have developed a system that allows us to index and deliver audio and video over the Internet. The system has been in continuous operation since March 1998 within the company. The design of our system differs from previous systems because 1) the indexing can be based on an annotation stream generated by robust transcript alignment, as well as closed captions, and 2) it is a distributed system that is designed for scalable, high performance, universal access through the World Wide Web. Extensive tests of the system show that it achieves a performance level required for Internet-wide delivery. This paper discusses our approach to the problem, the design requirements, the system architecture, and performance figures. It concludes by showing how the next generation of annotations from speech recognition and computer vision can be incorporated into the system.

Methods, Standards and Tools, 2005

By the end of the last century the question was not whether digital archives are technically and economically viable, but rather how digital archives would be efficient and informative. In this framework, different scientific fields such as, on the one hand, development of database management systems, and, on the other hand, processing and analysis of multimedia data, as well as artificial and computational intelligence methods, have observed a close cooperation with each other during the past few years. The attempt has been to develop intelligent and efficient human-computer interaction systems, enabling the user to access vast amounts of heterogeneous information, stored in different sites and archives.

Signal Processing: Image Communication, 2013

In this paper, a complete solution for search and retrieval of rich multimedia content over modern databases is presented. The framework proposed in this paper combines the advantages of multimodal search with those of annotation propagation into a unified system. Moreover, an effective technique, which is appropriate for large-scale indexing, is adopted, extended and integrated to the proposed framework so as to achieve optimized search and retrieval of rich media content even from large-scale databases.

2000

This paper describes various methods and approaches for language-based multimedia information retrieval, which have been developed in the projects POP-EYE and OLIVE and which will be developed further in the MUMIS project. All of these project aim at supporting automated indexing of video material by use of human language technologies. Thus, in contrast to image or sound-based retrieval methods, where both the query language and the indexing methods build on non-linguistic data, these methods attempt to exploit advanced text retrieval technologies for the retrieval of non-textual material. While POP-EYE was building on subtitles or captions as the prime language key for disclosing video fragments, OLIVE is making use of speech recognition to automatically derive transcriptions of the sound tracks, generating time-coded linguistic elements which then serve as the basis for text-based retrieval functionality.

Informatica (lithuanian Academy of Sciences), 2003

This paper describes the role advanced natural language processing (NLP) and especially information extraction (IE) can play for multimedia applications. As an example of such an application, we present an approach dealing with the automatic conceptual indexing of multimedia documents, which subsequently can be searched by semantic categories instead of key words. A novelty of the approach is to exploit

2012

An ever-growing amount of digitized content urges libraries and archives to integrate new media types from a large number of origins such as publishers, record labels and film archives, into their existing collections. This is a challenging task, since the multimedia content itself as well as the associated metadata is inherently heterogeneous—the different sources lead to different data structures, data quality and trustworthiness. This paper presents the contentus approach J. Nandzik (B) · N. Flores-Herr Acosta Consult GmbH, Zeißelstraße 15 HH, 60318 Frankfurt am Main, Germany e-mail: [email protected] N. Flores-Herr e-mail: [email protected] B. Litz · A. Löhden Deutsche Nationalbibliothek, Informationstechnik, Adickesallee 1, 60322 Frankfurt am Main, Germany B. Litz e-mail: [email protected] A. Löhden e-mail: [email protected] I. Konya · D. Baum · A. Bergholz Fraunhofer IAIS, Schloss Birlinghoven, 53754 Sankt Augustin, Germany I. Konya e-mail: [email protected]....

Lecture Notes in Computer Science, 2002

The indexing and retrieval of multimedia items is difficult due to the semantic gap between the user's perception of the data and the descriptions we can derive automatically from the data using computer vision, speech recognition, and natural language processing. In this contribution we consider the nature of the semantic gap in more detail and show examples of methods that help in limiting the gap. These methods can be automatic, but in general the indexing and retrieval of multimedia items should be a collaborative process between the system and the user. We show how to employ the users' interaction.

2003

Abstract MUVIS is a series of CBIR systems. The first one has been developed in late 90s to support indexing and retrieval in large image databases using visual and semantic features such as color, texture and shape. During recent years, MUVIS has been reformed to become a PC-based framework, which supports indexing, browsing and querying of various multimedia types such as audio, video, audio/video interlaced and several image formats.

1999

In this paper, we present a system that combines independent feature detector programs with multimedia database technology to provide a semantic rich index to multimedia data items on the World Wide Web.

2009

Currently the media production domain lacks efficient ways to organize and search for media assets. Ontology based applications have been identified as a viable solution to this problem, however, sometimes being too complex for non-experienced users. We present the SALERO Intelligent Media Annotation & Search system which provides an integrated view onto results retrieved from different search engines. Furthermore, it offers a powerful, yet user-friendly Web-based environment to organize and search for media assets.

Lecture Notes in Computer Science, 1999

The Cambridge Research Laboratory was founded in 1987 to advance the state of the art in both core computing and human-computer interaction, and to use the knowledge so gained to support the Company's corporate objectives. We believe this is best accomplished through interconnected pursuits in technology creation, advanced systems engineering, and business development. We are actively investigating scalable computing; mobile computing; vision-based human and scene sensing; speech interaction; computer-animated synthetic persona; intelligent information appliances; and the capture, coding, storage, indexing, retrieval, decoding, and rendering of multimedia data. We recognize and embrace a technology creation model which is characterized by three major phases:

Library Hi Tech, 2014

Purpose – The overwhelming speed and scale of digital media production greatly outpace conventional indexing methods by humans. The management of Big Data for e-library speech resources requires an automated metadata solution. The paper aims to discuss these issues. Design/methodology/approach – A conceptual model called semantic ontologies for multimedia indexing (SOMI) allows for assembly of the speech objects, encapsulation of semantic associations between phonic units and the definition of indexing techniques designed to invoke and maximize the semantic ontologies for indexing. A literature review and architectural overview are followed by evaluation techniques and a conclusion. Findings – This approach is only possible because of recent innovations in automated speech recognition. The introduction of semantic keyword spotting allows for indexing models that disambiguate and prioritize meaning using probability algorithms within a word confusion network. By the use of AI error-t...

2000

2002

This paper discusses our work on information extraction (IE) from multi-lingual, multi-media, multi-genre Language Resources, in a domain where there are many different event types. This work is being carried out in the context of MUMIS, an EU-funded project that aims at the development of basic technology for the creation of a composite index from multiple and multi-lingual sources. Our approach to IE relies on a £nite state machinery provided by GATE, a General Architecture for Text Engineering, pipelined with full syntactic analysis and discourse interpretation implemented in Prolog.

Multimedia Tools and Applications, 2010

Multimedia analysis and reuse of raw un-edited audio visual content known as rushes is gaining acceptance by a large number of research labs and companies. A set of research projects are considering multimedia indexing, annotation, search and retrieval in the context of European funded research, but only the FP6 project RUSHES is focusing on automatic semantic annotation, indexing and retrieval of raw and un-edited audio-visual content. Even professional content creators and providers as well as home-users are dealing with this type of content and therefore novel technologies for semantic search and retrieval are required. In this paper, we present a summary of the most relevant achievements of the RUSHES Multimed Tools Appl (2010) 48:23-49 project, focusing on specific approaches for automatic annotation as well as the main features of the final RUSHES search engine.