Constraint Techniques for Authoring Multimedia Documents

2009

Multimedia presentations are generally specified in terms of spatial and temporal relations between the media objects involved. Creation of these presentations, and interactions with them during their play-out, require an effective mechanism for handling the specifications dynamically. Further, these specifications could result in inconsistencies, which need to be checked and resolved. Our main contribution is the design of an algorithm which aids in resolving these consistency issues dynamically and efficiently. The other salient features of our approach are a new composite spatio-temporal operator and an effective relaxation policy. Thus, this paper presents an interactive multimedia presentation mechanism, which maintains a consistent and complete set of constraints during authoring and play-out of the presentation.

2001

We introduce a methodology for the high-level specification and decentralized coordination of temporal interdependencies among multimedia document objects. Such methodology encompasses a three-step process comprising (i) the design of multimedia presentation scenes by means of a fuzzy descriptive plan; (ii) the parsing of such layout to classify the multimedia entities that compose the scenes and to check the consistency of temporal relationships among them; and (iii) the generation of event-driven time and action managers as distributed mechanisms for the orchestration of the elements presentation. This approach centers around a novel multimedia synchronization model based on fuzzy sets and software components concepts.

1992

Constraints provide a natural formalism for user-interface design and graphical layout. Recent results and algorithms from symbolic computation and geometry provide new techniques to manipulate linear arithmetic constraints. We show how these results can be applied to interactive graphical user-interfaces and how they extend the capabilities of previous interactive constraint-based user interface systems. We propose an architecture for such systems based on these techniques .

2006

Trends in Constraint Programming, 2007

2005

Abstract Synchronized multimedia applications play an important role in a digital library environment, since they allow one to efficiently disseminate knowledge among differently skilled users through an approach, which is more direct than the classic'static'documents. In this paper, we propose a new authoring approach based on an innovative presentation structure and a new class of content-based constraints.

We introduce a methodology for the high-level specification and decentralized coordination of temporal interdependencies among multimedia document objects. Such methodology encompasses a three-step process comprising (i) the design of multimedia presentation scenes by means of a fuzzy descriptive plan; (ii) the parsing of such layout to classify the multimedia entities that compose the scenes and to check the consistency of temporal relationships among them; and (iii) the generation of event-driven time and action managers as distributed mechanisms for the orchestration of the elements presentation. This approach centers around a novel multimedia synchronization model based on fuzzy sets and software components concepts.

1999

The developing field of multimedia opens up a whole new class of applications for constraint techniques. The aim of this talk is twofold: show innovative applications of existing constraints techniques, and also show how the new landscape of multimedia raises new issues pertaining to constraint research and development. I will first outline the main results in so-called automatic harmonization, i.e. the use of constraints to generate music according to given inputs such as melodies or bass lines. I will then show how constraint perturbation techniques can be used to build interactive listening systems, especially for controlling sound source spatialization. Then I show an application of constraint satisfaction for creating music programs satisfying user preferences. I illustrate this idea on a prototype music program generation system.

1997

Abstract. This paper explores a formalism for describing a wide class of multimedia document constraints. We outline the requirements on temporal logic specification that arise from the multimedia documents application area. In particular, we highlight a canonical document example. Then we present the temporal logic formalism that we use.

TRIP systems are tools for visualization and animation. They are based on a constraint-based model of bi-directional translation between abstract data and pictorial data. Using these systems, programmers can visualize abstract data, and animate various algorithms and processes, simply by providing a declarative mapping rule. This paper presents this model for visualization and animation, focusing on the use of constraints, and also presents the TRIP systems with examples of visualization and animation.

Lecture Notes in Computer Science, 1994

Local propagation is often used in graphical user interfaces to solve constraint systems that describe structures and layouts of gures. However, algorithms based on local propagation cannot solve simultaneous constraint systems because local propagation must solve constraints individually. W e propose the`DETAIL' algorithm, which e ciently solves systems of constraints with strengths, even if they must be solved simultaneously, b y`dividing' them as much as possible. In addition to multi-way constraints, it handles various other types of constraints, for example, constraints solved with the least squares method. Furthermore, it uni es the treatment of di erent t ypes of constraints in a single system. We implemented a prototype constraint solver based on this algorithm, and evaluated its performance.

ACM Transactions on Multimedia Computing, Communications, and Applications, 2005

A traditional static document has a spatial layout that specifies where objects in the document appear. Because multimedia documents incorporate time, they also require a temporal layout, or schedule, that specifies when events in the document occur. This article argues that multimedia document systems should provide mechanisms for automatically producing temporal layouts for documents. The major advantage of this approach is that it makes it easier for authors to create and modify multimedia documents. This article revisits our 1993 framework for understanding automatic temporal formatters and explores the basic issues surrounding them. It also describes the Firefly multimedia document system, which was developed in 1992 to test the potential of automatic temporal formatting. Using our original framework, the paper reviews a representative sample of recent automatic document formatters. This analysis validates the basic framework and demonstrates the progress of the field in the intervening decade. A discussion of potential extensions to the framework is included.

Proceedings of the sixth ACM international conference on Multimedia - MULTIMEDIA '98, 1998

Proceedings of the fourth ACM international conference on Multimedia - MULTIMEDIA '96, 1996

Creating high-quality multimedia presentations requires much skill, time, and effort. This is particularly true when temporal media, such as speech and animation, are involved. We describe the design and implementation of a knowledge-based system that generates customized temporal multimedia presentations. We provide an overview of the system's architecture, and explain how speech, written text, and graphics are generated and coordinated. Our emphasis is on how temporal media are coordinated by the system through a multi-stage negotiation process. In negotiation, media-specific generation components interact with a novel coordination component that solves temporal constraints provided by the generators. We illustrate our work with a set of examples generated by the system in a testbed application intended to update hospital caregivers on the status of patients who have undergone a cardiac bypass operation.

1986

WILLIAM JAY LELER Specification and Generation of Constraint Satisfaction Systems (Under the direction of Bharat Jayaraman.) Constraint languages are declarative languages that have been used in various applications such as simulation, modeling, and graphics. Unfortunately, despite their benefits, existing constraint languages tend to be application-specific, have limited extensibility, and are difficult to implement. This dissertation presents a general-purpose computer language that makes it much easier to describe and implement constraint satisfaction systems. This language, called Bertrand, supports a rule-based programming methodology and also includes a form of abstract datatype. It is implemented using a new inference mechanism called augmented term rewriting, which is an extension of standard term rewriting. Using rules, a Bertrand programmer can describe new constraint satisfaction mechanisms, including equation solvers. Rules can also be used to define new types of objects and new constraints on these objects. Augmented term rewriting uses these rules to find a model that satisfies a set of user-specified constraints. The simple semantics of augmented term rewriting makes it possible to take advantage of several known techniques for optimizing execution speed, such as fast pattern matching, compilation, constant propagation, and concurrency. Consequently, Bertrand programs can be executed efficiently using an interpreter or compiled to run on a conventional or parallel processor. This dissertation surve;ys existing constraint satisfaction techniques and languages, and shows how they can be implemented in Bertrand. It also gives a precise operational semantics for augmented term rewriting. Techniques for efficient execution, including interpretation and compilation, are presented. Finally, examples are given using Bertrand to solve problems in algebra such as word problems and computer aided engineering, and problems in graphics, such as computer aided design, illustration, and mapping.