Semantic Query Execution in a Video Database System

Lecture Notes in Computer Science, 2000

A constraint of existing content-based video data models is that each modeled semantic description must be associated with time intervals exactly within which it happens and semantics not related to any time interval are not considered. Consequently, users are provided with limited query capabilities. This paper is aimed at developing a novel model with two innovations: (1) Semantic contents not having related time information can be modeled as ones that do (2) Not only the temporal feature of semantic descriptions, but also the temporal relationships among themselves are components of the model. The query system is by means of reasoning on those relationships. To support users' access, a video algebra and a video calculus as formal query languages, which are based on semantic relationship reasoning, are also presented.

Multimedia Tools and Applications, 2005

With the advances in information technology, the amount of multimedia data captured, produced, and stored is increasing rapidly. As a consequence, multimedia content is widely used for many applications in today's world, and hence, a need for organizing this data, and accessing it from repositories with vast amount of information has been a driving stimulus both commercially and academically. In compliance with this inevitable trend, first image and especially later video database management systems have attracted a great deal of attention, since traditional database systems are designed to deal with alphanumeric information only, thereby not being suitable for multimedia data. In this paper, a prototype video database management system, which we call BilVideo, is introduced. The system architecture of BilVideo is original in that it provides full support for spatio-temporal queries that contain any combination of spatial, temporal, object-appearance, external-predicate, trajectory-projection, and similaritybased object-trajectory conditions by a rule-based system built on a knowledge-base, while utilizing an objectrelational database to respond to semantic (keyword, event/activity, and category-based), color, shape, and texture queries. The parts of BilVideo (Fact-Extractor, Video-Annotator, its Web-based visual query interface, and its SQL-like textual query language) are presented, as well. Moreover, our query processing strategy is also briefly explained.

2003

Abstract The BilVideo video database management system provides integrated support for spatiotemporal and semantic queries for video. A knowledge base, consisting of a fact base and a comprehensive rule set implemented in Prolog, handles spatio-temporal queries. These queries contain any combination of conditions related to direction, topology, 3D relationships, object appearance, trajectory projection, and similarity-based object trajectories.

Workshop on Multimedia Information Systems, 2002

The most useful environments for advancing research and development in video databases are those that provide complete video database management, including (1) video preprocessing for content representation and indexing, (2) storage management for video, metadata and indices, (3) image and semantic -based query processing, (4) real- time buffer management, and (5) continuous media streaming. Such environments support the entire process

Multimedia Tools and Applications, 2000

Users of browsing applications often have vague information needs which can only be described in conceptual terms. Therefore, a video browsing system must accept conceptual queries for preselection and offer mechanisms for interactive inspection of the result set by the user. In this paper, we describe a MM-DBMS that we extended with the following components: Our retrieval engine calculates relevance

2002

Advances in compression techniques, decreasing cost of storage, and high-speed transmission have facilitated the way video is created, stored and distributed. As a consequence, video is now being used in many application areas. The increase in the amount of video data deployed and used in today's applications not only caused video to draw more attention as a multimedia data type, but also led to the requirement of efficient management of video data. Management of video data paved the way for new research areas, such as indexing and retrieval of videos with respect to their spatio-temporal, visual and semantic contents. In this paper, semantic content of video is studied, where video metadata, activities, actions and objects of interest are considered within the context of video semantic content. A data model is proposed to model video semantic content, which is extracted from video data by a video annotation tool. The work in this paper constitutes a part of a video database system to provide support for semantic queries.

2001

Video database and video on demand represent important applications of the evolving Global Information Infrastructure. Conventional database management systems (DBMS) are not well-suited for video querying and retrieval because they are designed to operate on alphanumeric data where selection conditions are easier to specify. Defining similarity between video data is difficult since the similarity involves semantics of the video, which is inherent in the data itself. We are planning to integrate two approaches to modeling, indexing and querying video data. The first one is based on textual annotations and the second is based on object motion and digital video processing techniques, in order to support content based and intelligent access. In addition to this novel data model, we also plan to develop a s ystem prototype that can synergistically use multiple media modes to retrieve data using fuzzy information fusion.

Proceedings of the first ACM international workshop on Multimedia databases - MMDB 2003, 2003

The increased use of video data sets for multimedia-based applications has created a demand for strong video database support, including efficient methods for handling the contentbased query and retrieval of video data. Video query processing presents significant research challenges, mainly associated with the size, complexity and unstructured nature of video data. A video query processor must support video operations for search by content and streaming, new query types, and the incorporation of video methods and operators in generating, optimizing and executing query plans. In this paper, we address these query processing issues in two contexts, first as applied to the video data type and then as applied to the stream data type. We present the query processing functionality of the VDBMS video database management system, which was designed to support a full range of functionality for video as an abstract data type. We describe two query operators for the video data type which implement the rank-join and stop-after algorithms. As videos may be considered streams of consecutive image frames, video query processing can be expressed as continuous queries over video data streams. The stream data type was therefore introduced into the VDBMS system, and system functionality was extended to support general data streams. From this viewpoint, we present an approach for defining and processing streams, including video, through the query execution engine. We describe the implementation of several algorithms for video query processing expressed as continuous queries over video streams, such as fast forward, region-based blurring and left outer join. We include a description of the window-join algorithm as a core operator for continuous query systems, and discuss shared execution as an optimization approach for stream query processing.

2002

In this paper we describe the first full implementation of a content-based indexing and retrieval system for MPEG-2 and MPEG-4 videos. We consider a video as a collection of spatiotemporal segments called video objects; each video object is a sequence of video object planes. A set of representative video object planes is used to index each video object. During the database population, the operator, using a semi-automatic outlining tool we developed, manually selects video objects and insert some semantical information.

Information Sciences, 2002

We propose a novel architecture for a video database system incorporating both spatio-temporal and semantic (keyword, event/activity and category-based) query facilities. The originality of our approach stems from the fact that we intend to provide full support for spatio-temporal, relative object-motion and similarity-based objecttrajectory queries by a rule-based system utilizing a knowledge-base while using an object-relational database to answer semantic-based queries. Our method of extracting and modeling spatio-temporal relations is also a unique one such that we segment video clips into shots using spatial relationships between objects in video frames rather than applying a traditional scene detection algorithm. The technique we use is simple, yet novel and powerful in terms of effectiveness and user query satisfaction: video clips are segmented into shots whenever the current set of relations between objects changes and the video frames, where these changes occur, are chosen as keyframes. The directional, topological and third-dimension relations used for shots are those of the keyframes selected to represent the shots and this information is kept, along with frame numbers of the keyframes, in a knowledge-base as Prolog facts. The system has a comprehensive set of inference rules to reduce the number of facts stored in the knowledge-base because a considerable number of facts, which otherwise would have to be stored explicitly, can be derived by rules with some extra effort.