Multimedia explorer: Content based multimedia exploration

2003

MUVIS is a series of CBIR systems. The first one has been developed in late 90s to support indexing and retrieval in large image databases using visual and semantic features such as color, texture and shape. During recent years, MUVIS has been reformed to become a PCbased framework, which supports indexing, browsing and querying of various multimedia types such as audio, video, audio/video interlaced and several image formats. MUVIS system allows real-time audio and video capturing, encoding by last generation codecs such as MPEG-4, H.263+, MP3 and AAC. It supports several audio/video file format such as AVI, MP4, MP3 and AAC. Furthermore, MUVIS system provides a welldefined interface for third parties to integrate their own feature extraction algorithms into the framework and for this reason it has recently been adopted by COST 211quat as COST framework for CBIR. In this paper, we describe the general system features with underlying applications and outline the main philosophy.

2002

Current advances in multimedia technology enable ease of capturing and encoding digital video. As a result, video data is rapidly growing and becoming very important in our life. It is because video can transfer a large amount of knowledge by providing combination of text, graphics, or even images. Despite the vast growth of video, the effectiveness of its usage is very limited due to the lack of a complete technology for the organization and retrieval of video data. To date, there is no "perfect" solution for a complete video data-management technology, which can fully capture the content of video and index the video parts according to the contents, so that users can intuitively retrieve specific video segments. We have found that successful content-based video datamanagement systems depend on three most important components: key-segments extraction, content descriptions and video retrieval. While it is almost impossible for current computer technology to perceive the content of the video to identify correctly its key-segments, the system can understand more accurately the content of a specific video type by identifying the typical events that happens just before or after the key-segments (specific-domainapproach). Thus, we have proposed a concept of customisable video segmentation module, which integrates the suitable segmentation techniques for the current type of video. The identified key-segments are then described using standard video content descriptions to enable content-based retrievals. For retrieval, we have implemented XQuery, which currently is the most recent XML query language and the most powerful compared to older languages, such as XQL and XML-QL.

1999

In this paper, we describe description schemes (DSs) for image, video, multimedia, home media, and archive content proposed to the MPEG-7 standard. MPEG-7 aims to create a multimedia content description standard in order to facilitate various multimedia searching and "ltering applications. During the design process, special care was taken to provide simple but powerful structures that represent generic multimedia data. We use the extensible markup language (XML) to illustrate and exemplify the proposed DSs because of its interoperability and #exibility advantages. The main components of the image, video, and multimedia description schemes are object, feature classi"cation, object hierarchy, entity-relation graph, code downloading, multi-abstraction levels, and modality transcoding. The home media description instantiates the former DSs proposing the 6-W semantic features for objects, and 1-P physical and 6-W semantic object hierarchies. The archive description scheme aims to describe collections of multimedia documents, whereas the former DSs only aim at individual multimedia documents. In the archive description scheme, the content of an archive is represented using multiple hierarchies of clusters, which may be related by entity-relation graphs. The hierarchy is a speci"c case of entity-relation graph using a containment relation. We explicitly include the hierarchy structure in our DSs because it is a natural way of de"ning composite objects, a more e$cient structure for retrieval, and the representation structure used in MPEG-4. We demonstrate the feasibility and the e$ciency of our description schemes by presenting applications that already use the proposed structures or will greatly bene"t from their use. These applications are the visual apprentice, the AMOS-search system, a multimedia broadcast news browser, a storytelling system, and an image meta-search engine, MetaSEEk.

Faculty of Science and Technology, 2002

Current advances in multimedia technology enable ease of capturing and encoding digital video. As a result, video data is rapidly growing and becoming very important in our life. It is because video can transfer a large amount of knowledge by providing combination of text, graphics, or even images. Despite the vast growth of video, the effectiveness of its usage is very limited due to the lack of a complete technology for the organization and retrieval of video data. To date, there is no "perfect" solution for a complete video data-management technology, which can fully capture the content of video and index the video parts according to the contents, so that users can intuitively retrieve specific video segments. We have found that successful content-based video datamanagement systems depend on three most important components: key-segments extraction, content descriptions and video retrieval. While it is almost impossible for current computer technology to perceive the content of the video to identify correctly its key-segments, the system can understand more accurately the content of a specific video type by identifying the typical events that happens just before or after the key-segments (specific-domainapproach). Thus, we have proposed a concept of customisable video segmentation module, which integrates the suitable segmentation techniques for the current type of video. The identified key-segments are then described using standard video content descriptions to enable content-based retrievals. For retrieval, we have implemented XQuery, which currently is the most recent XML query language and the most powerful compared to older languages, such as XQL and XML-QL.

Retrieval in current multimedia databases is usually limited to browsing and searching based on low-level visual features and explicit textual descriptors. Semantic aspects of visual information are mainly described in full text attributes or mapped onto specialized, application specific description schemes. Result lists of queries are commonly represented by textual descriptions and single key frames. This approach is valid for text documents and images, but is often insufficient to represent video content in a meaningful way. In this paper we present a multimedia retrieval framework focusing on video objects, which fully relies on the MPEG-7 standard as information base. It provides a content-based retrieval interface which uses hierarchical content-based video summaries to allow for quick viewing and browsing through search results even on bandwidth limited Web applications. Additionally semantic meaning about video content can be annotated based on domain specific ontologies, enabling a more targeted search for content. Our experiences and results with these techniques will be discussed in this paper.

2004

Retrieval in current multimedia databases is usually limited to browsing and searching based on low-level visual features and explicit textual descriptors. Semantic aspects of visual information are mainly described in full text attributes or mapped onto specialized, application specific description schemes. Result lists of queries are commonly represented by textual descriptions and single key frames. This approach is valid for text documents and images, but is often insufficient to represent video content in a meaningful way. In this paper we present a multimedia retrieval framework focusing on video objects, which fully relies on the MPEG-7 standard as information base. It provides a content-based retrieval interface which uses hierarchical content-based video summaries to allow for quick viewing and browsing through search results even on bandwidth limited Web applications. Additionally semantic meaning about video content can be annotated based on domain specific ontologies, enabling a more targeted search for content. Our experiences and results with these techniques will be discussed in this paper.

The current level of request for applications involving content-based video coding and video information retrieval is increasing the relevance of systems able to somehow 'understand' the content of video sequences. This type of systems will enable the provision of services where enhanced user interaction with the visual content is supported. Such an object-based video coder is being specified by ISO under the MPEG-4 project [1], while the standardization of video description capabilities to support content-based video indexing and retrieval is being considered by MPEG-7 [2]. In this paper, we discuss the issue of video content identification and characterization, and the importance of providing the means for the user to interact with the analysis process so that the achieved results can have a meaningful and powerful semantic value.

… Indexing (CBMI), 2011 …, 2011

In this paper we present two interactive systems for video search and browsing; one is a web application based on the Rich Internet Application paradigm, designed to obtain the levels of responsiveness and interactivity typical of a desktop application, while the other exploits multi-touch devices to implement a multi-user collaborative application. Both systems use the same ontology-based video search engine, that is capable of expanding user queries through ontology reasoning, and let users to search for specific video segments that contain a semantic concept or to browse the content of video collections, when it's too difficult to express a specific query.

DELOS Research Activities 2005, 2005

The increasing availability of high-speed wired and wireless networks as well as the development of a new generation of powerful (mobile) end-user devices like PDAs or cell phones leads to new ways of multimedia resource consumption. At the same time, new standards like MPEG-7/21 have become available, allowing us the enrichment of media content with semantic content annotations, which in turn facilitates new forms of multimedia experience, like search on specific topics or semantic-based content selection, filtering, ...

2002

Abstract. The task of content–based retrieval is to provide users with the multimedia documents that best match their wishes. This process is not free of uncertainty; the role of the user profile is to remove a part of this uncertainty, using the information it contains concerning the user's preferences, and thus improve the precision. For this aim MPEG-7 introduced description schemes representing users' preferences and tools supporting user interaction.

2002

The indexing and retrieval of multimedia items is difficult due to the semantic gap between the user’s perception of the data and the descriptions we can derive automatically from the data using computer vision, speech recognition, and natural language processing. In this contribution we consider the nature of the semantic gap in more detail and show examples of methods that help in limiting the gap. These methods can be automatic, but in general the indexing and retrieval of multimedia items should be a collaborative process between the system and the user. We show how to employ the user’s interaction for limiting the semantic gap.