ONTOLOGY FOR SEMANTIC MULTIMEDIA WEB

Multimedia constitutes an interesting field of application for Semantic Web and Semantic Web reasoning, as the access and management of multimedia content and context depends strongly on the semantic descriptions of both. At the same time, multimedia resources constitute complex objects, the descriptions of which are involved and require the foundation on sound modeling practice in order to represent findings of low-and high level multimedia analysis and to make them accessible via Semantic Web querying of resources. This tutorial aims to provide a red thread through these different issues and to give an outline of where Semantic Web modeling and reasoning needs to further contribute to the area of semantic multimedia for the fruitful interaction between these two fields of computer science. 1

International Semantic Web Working Symposium (SWWS), 2001

For the past two years the Moving Pictures Expert Group (MPEG), a working group of ISO/IEC, have been developing , the "Multimedia Content Description Interface", a standard for describing multimedia content. The goal of this standard is to develop a rich set of standardized tools to enable both humans and machines to generate and understand audiovisual descriptions which can be used to enable fast efficient retrieval from digital archives (pull applications) as well as filtering of streamed audiovisual broadcasts on the Internet (push applications). MPEG-7 is intended to describe audiovisual information regardless of storage, coding, display, transmission, medium, or technology. It will address a wide variety of media types including: still pictures, graphics, 3D models, audio, speech, video, and combinations of these (e.g., multimedia presentations). MPEG-7 is due for completion in October 2001. At this stage MPEG-7 definitions (description schemes and descriptors) are expressed solely in XML Schema . XML Schema has been ideal for expressing the syntax, structural, cardinality and datatyping constraints required by MPEG-7. However it has become increasingly clear that in order to make MPEG-7 accessible, re-usable and interoperable with other domains then the semantics of the MPEG-7 metadata terms also need to be expressed in an ontology using a machine-understandable language. This paper describes the trials and tribulations of building such an ontology represented in RDF Schema and demonstrates how this ontology can be exploited and reused by other communities on the semantic web (such as TV-Anytime [6], , NewsML [8], museum, educational and geospatial domains) to enable the inclusion and exchange of multimedia content through a common understanding of the associated MPEG-7 multimedia content descriptions. • A Description Definition Language (DDL) to allow the creation of new Description Schemes and, possibly, Descriptors and to allows the extension and modification of existing Description Schemes; • System tools, to support multiplexing of descriptions, synchronization of descriptions with content, transmission mechanisms and coded representations (both textual and binary formats) for efficient storage and transmission, management and protection of intellectual property in MPEG-7 descriptions. XML Schema language has been chosen as the DDL [9] for specifying MPEG-7 descriptors and description schemes because of its ability to express the syntactic, structural, cardinality and datatyping constraints required by MPEG-7 and because it also provides the necessary mechanisms for extending and refining existing DSs and Ds. However it has recently become increasingly clear that there is also a need for a machine-understandable representation of the semantics associated with MPEG-7 DSs and Ds to enable the interoperability and integration of MPEG-7 with metadata descriptions from other domains. New metadata initiatives such as TV-Anytime [6], MPEG-21 [7], NewsML , and communities such as the museum, educational, medical and geospatial communities, want to combine MPEG-7 multimedia descriptions with new and existing metadata standards for simple resource discovery (Dublin Core [10]), rights management (INDECS [11]), geospatial (FGDC [12]), educational (GEM [13], IEEE LOM ) and museum (CIDOC CRM ) content, to satisfy their domain-specific requirements. In order to do this, there needs to be a common understanding of the semantic relationships between metadata terms from different domains. XML Schema provides little support for expressing semantic knowledge. RDF Schema provides us with a way to do this. The Resource Description Framework (RDF) is the accepted language of the semantic web due to its ability to express semantics and semantic relationships through class and property hierarchies. In this paper, we investigate the feasibility of expressing the semantics of MPEG-7 Descriptors (Ds) and Description Schemes (DSs) in an RDF Schema ontology. An earlier paper evaluated RDF Schema for video metadata representation (prior to the development of MPEG-7) and determined a number of limitations . In this paper we hope to ascertain whether those limitations still exist when representing the semantics of MPEG-7 DSs and Ds or whether they can be overcome -either by using the extra constraints provided by DAML+OIL [17] or through combining RDF Schema semantics with XML Schema encoding specifications in a complementary manner. Whilst manually building the RDF Schema for a core subset of MPEG-7, we also hope to be able to recognize patterns and hence determine automatic mechanisms for generating compatible RDF Schema definitions corresponding to the complete set of MPEG-7 XML Schema definitions. In Section 2 we describe the methodology, problems encountered and results of building an RDF Schema ontology for MPEG-7. In Section 3 we describe how the RDF Schema semantic definitions for MPEG-7 can be linked to their corresponding pre-existing XML Schema definitions (or recommended encodings). In Section 4 we describe how the MPEG-7 RDF Schema can be merged with RDF schemas from other domains to generate a single "super-ontology" called MetaNet. Expressed in DAML+OIL , MetaNet can be used to provide common semantic understanding between domains. Finally we illustrate how this super-ontology can be used to enable the co-existence of interoperability, extensibility and diversity within metadata descriptions generated by integrating metadata terms from different domains.

IEEE Transactions on Circuits and Systems for Video Technology, 2003

A core ontology is one of the key building blocks necessary to enable the integration of information from diverse multimedia sources. A complete and extensible ontology that expresses the basic concepts that are common across a variety of domains and media types and that can provide the basis for specialization into domain-specific concepts and vocabularies, is essential for well-defined mappings between domain-specific knowledge representations (i.e., metadata vocabularies) and the subsequent building of a variety of services such as cross-domain searching, tracking, browsing, data mining and knowledge acquisition. As more and more communities develop metadata application profiles which combine terms from multiple vocabularies (e.g., Dublin Core, MPEG-7, MPEG-21, CIDOC/CRM, FGDC, IMS) such a core ontology will provide a common understanding of the basic entities and relationships which is essential for semantic interoperability and the development of additional services based on deductive inferencing. In this paper we first propose such a core ontology (the ABC model) which was developed in response to a need to integrate information from multiple genres of multimedia content within digital libraries and archives. Although the MPEG-21 RDD was influenced by the ABC model and is based on a model extremely similar to ABC, we believe that it is important to define a separate and domain-independent top-level extensible ontology for scenarios in which either MPEG-21 is irrelevant or to enable the attachment of ontologies from communities external to MPEG, for example, the museum domain (CIDOC/CRM) or the bio-medical domain (ON9.3). We then evaluate the ABC model's ability to mediate and integrate between multimedia metadata vocabularies by illustrating how it can provide the foundation to facilitate semantic interoperability between MPEG-7, MPEG-21 and other domain-specific metadata vocabularies. By expressing the semantics of both MPEG-7 and MPEG-21 metadata terms in RDF Schema/DAML+OIL (and eventually the Web Ontology Language (OWL)) and attaching the MPEG-7 and MPEG-21 class and property hierarchies to the appropriate top-level classes and properties of the ABC model, we have defined a single distributed machine-understandable ontology. The resulting distributed machineprocessable ontology provides semantic knowledge which is non-existent within declarative XML schemas or XML-encoded metadata descriptions. Finally, in order to illustrate how such an ontology will contribute to the interoperability of data and services across the entire multimedia content delivery chain, we describe a number of valuable services which have been developed or could potentially be developed using the resulting merged ontologies.

2010

Diversity of standards, languages, protocols, and hardware components leads to important incompatibility issues when designing and developing multiplatform multimedia systems. Furthermore, user and community requirements and preferences should be taken into account when instantiating and configuring these kind of systems. This work introduces an ontology framework for distributed multimedia systems design and deployment driven by user requirements. Case studies illustrating how distributed multimedia systems can be dynamically deployed are presented in order to demonstrate the advantages of using this framework.

In recent years, along with the expansion of Web 2.0 and social networks, an extreme growth of multimedia content on the Web is registered. That multimedia content is mostly in the form of images and videos. To enable enhanced use, reuse and retrieval of multimedia content from the Web, that content needs to be annotated. Several multimedia metadata standards and a number of vocabularies commonly used for annotating multimedia content exist today. Semantic Web technologies, like RDF and ontologies, provide well-defined meaning for the multimedia content, enabling better processing of their annotations by computers and applications. Formal language OWL, along with its sublanguages, is used for defining ontologies on the Semantic Web. In this paper a brief overview of ontologies in general and selected specialized multimedia ontologies that can be used for semantically rich multimedia annotation is presented.

Proc. of the European …, 2004

2nd European Workshop on the Integration of Knowledge, Semantics and Digital Media Technology (EWIMT 2005), 2005

The Semantic Web is about adding formal structure and explicit semantics to web content for the purpose of more efficient information management and access by both humans and computers. Through the use of machine understandable semantics, web resources become much easier and more readily accessible. Populating the Semantic Web with multimedia metadata requires for appropriate technologies to support both the analysis (metadata extraction) and the annotation (metadata creation) processes. The media-and contentspecific semantics of multimedia resources pose great challenges in terms of capturing and manipulating the two layer semantics given the available knowledge representation formalisms. The Knowledge Web project, advancing ontology technology research efforts and ensuring transferring the semantic web from academia to industry, has provided solutions to some of the most challenging multimedia related problems.

International Journal of Web Engineering and Technology, 2005

Cultural institutions, broadcasting companies, academic, scientific and defence organisations are producing vast quantities of digital multimedia content. With this growth in audiovisual material comes the need for standardised representations encapsulating the rich semantic meaning required to enable the automatic filtering, machine processing, interpretation and assimilation of multimedia resources. Additionally generating high-level descriptions is difficult and manual creation is expensive although significant progress has been made in recent years on automatic segmentation and low-level feature recognition for multimedia. Within this paper we describe the application of semantic web technologies to enable the generation of high-level, domain-specific, semantic descriptions of multimedia content from low-level, automatically-extracted features. By applying the knowledge reasoning capabilities provided by ontologies and inferencing rules to large, multimedia data sets generated by scientific research communities, we hope to expedite solutions to the complex scientific problems they face.

2014

Today’s major problem in consumption of multimedia content from the Web is the extremely large volume of multimedia content in variou s forms on the Web, which keeps on rapidly growing. Another problem is that one part of that multimedia content is not annotated ; therefore it is very hard to find and reuse such content. The other part of multimedia content is described manually, hence th ose annotations may be too subjective or inaccurate, and may be lacking in formal semantics. This results in the need for efficient semantic annotation, so that computers and applications can easily process those metadata for reuse and retrieval of multime dia content. This paper presents ontologies in general as part of Semantic Web an d s pecific ontologies used for multimedia annotation . Comp arison of the most commo n ly used multimedia ontologies and their main features is provided in this paper. The se multimedia ontologies can be used for creating high qua lity and semantically rich multime...