A System for Distributed Multi-camera Capture and Processing
Peter Schubel2010, 2010 Conference on Visual Media Production
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
This contribution describes a distributed multi-camera capture and processing system for real-time media production applications. Its main design purpose is to allow prototyping of distributed processing algorithms for free-viewpoint applications, but the concept can be adapted to other (multicamera) applications.
Related papers
Wireless Software Synchronization of Multiple Distributed Cameras
2019 IEEE International Conference on Computational Photography (ICCP), 2019
We present a method for precisely time-synchronizing the capture of image sequences from a collection of smartphone cameras connected over WiFi. Our method is entirely software-based, has only modest hardware requirements, and achieves an accuracy of less than 250 µs on unmodified commodity hardware. It does not use image content and synchronizes cameras prior to capture. The algorithm operates in two stages. In the first stage, we designate one device as the leader and synchronize each client device's clock to it by estimating network delay. Once clocks are synchronized, the second stage initiates continuous image streaming, estimates the relative phase of image timestamps between each client and the leader, and shifts the streams into alignment. We quantitatively validate our results on a multi-camera rig imaging a high-precision LED array and qualitatively demonstrate significant improvements to multi-view stereo depth estimation and stitching of dynamic scenes. We release as open source libsoftwaresync, an Android implementation of our system, to inspire new types of collective capture applications.
Interconnected network of cameras
Sensors, Cameras, and Systems for Industrial and Scientific Applications XIV, 2013
The real-time development of multi-camera systems is a great challenge. Synchronization and large data rates of the cameras adds to the complexity of these systems as well. The complexity of such system also increases as the number of their incorporating cameras increases. The customary approach to implementation of such system is a central type, where all the raw stream from the camera are first stored then processed for their target application. An alternative approach is to embed smart cameras to these systems instead of ordinary cameras with limited or no processing capability. Smart cameras with intra and inter camera processing capability and programmability at the software and hardware level will offer the right platform for distributed and parallel processing for multicamera systems real-time application development. Inter camera processing requires the interconnection of smart cameras in a network arrangement. A novel hardware emulating platform is introduced for demonstrating the concept of the interconnected network of cameras. A methodology is demonstrated for the interconnection network of camera construction and analysis. A sample application is developed and demonstrated.
<title>A generic software-framework for distributed, high-performance processing of multi-view video</title>
Real-Time Image Processing 2007, 2007
This paper presents a software framework providing a platform for parallel and distributed processing of video data on a cluster of SMP computers. Existing video-processing algorithms can be easily integrated into the framework by considering them as atomic processing tiles (PTs). PTs can be connected to form processing graphs that model the data flow of a specific application. This graph also defines the data dependencies that determine which tasks can be computed in parallel. Scheduling of the tasks in this graph is carried out automatically using a pool-of-tasks scheme. The data format that can be processed by the framework is not restricted to image data, such that also intermediate data, like detected feature points or object positions, can be transferred between PTs. Furthermore, the processing can optionally be carried out efficiently on special-purpose processors with separate memory, since the framework minimizes the transfer of data. Finally, we describe an example application for a multi-camera view-interpolation system that we successfully implemented on the proposed framework.
Temporal synchronization of video sequences in theory and in practice
2005
In this work, we present a formalization of the video synchronization problem that exposes new variants of the problem that have been left unexplored to date. We also present a novel method to temporally synchronize multiple stationary video cameras with overlapping views that: 1) does not rely on certain scene properties, 2) suffices for all variants of the synchronization problem exposed by the theoretical disseration, and 3) does not rely on the trajectory correspondence problem to be solved apriori. The method uses a two stage approach that first approximates the synchronization by tracking moving objects and identifying inflection points. The method then proceeds to refine the estimate using a consensus based matching heuristic to find moving features that best agree with the pre-computed camera geometries from stationary image features. By using the fundamental matrix and the trifocal tensor in the second refinement step we are able to improve the estimation of the first step and handle a broader range of input scenarios and camera conditions.
A Temporal Synchronization Mechanism for Real-Time Distributed Continuous Media
Signal Processing and Multimedia Applications, 2006
The preservation of temporal relations for real-time distributed continuos media is a key issue for emerging multimedia applications, such as Tele-Immersion and Tele-Engineering. Although several works try to model and execute distributed continuous media scenarios, they are far from resolving the problem. The present paper proposes a viable solution based on the identification of logical dependencies. Our solution considers two
Generic Framework for Parallel and Distributed Processing of Video-Data
Lecture Notes in Computer Science, 2006
This paper presents a software framework providing a platform for parallel and distributed processing of video data on a cluster of SMP computers. Existing video-processing algorithms can be easily integrated into the framework by considering them as atomic processing tiles (PTs). PTs can be connected to form processing graphs that model the data flow. Parallelization of the tasks in this graph is carried out automatically using a pool-of-tasks scheme. The data format that can be processed by the framework is not restricted to image data, such that also intermediate data, like detected feature points, can be transferred between PTs. Furthermore, the processing can be carried out efficiently on special-purpose processors with separate memory, since the framework minimizes the transfer of data. We also describe an example application for a multi-camera view-interpolation system that we successfully implemented on the proposed framework.
Synchronization of Video Sequences from Free-Moving Cameras
2007
We present a new method for the synchronization of a pair of video sequences and the spatial registration of all the temporally corresponding frames. This is a mandatory step to perform a pixel wise comparison of a pair of videos. Several proposals for video matching can be found in the literature, with a variety of applications like object detection, visual sensor fusion, high dynamic range and action recognition. The main contribution of our method is that it is free from three common restrictions assumed in previous works. First, it does not impose any condition on the relative position of the two cameras, since they can move freely. Second, it does not assume a parametric temporal mapping relating the time stamps of the two videos, like a constant or linear time shift. Third, it does not rely on the complete trajectories of image features (points or lines) along time, something difficult to obtain automatically in general. We present our results in the context of the comparison of videos captured from a camera mounted on moving vehicles.
Multimedia Synchronization
SpringerReference
This paper presents a hierarchical synchronization model for the description of the time relations and regimes necessary for the presentation of multimedia or animated data which have either natural or implied time dependencies. The model generalizes some previous multimedia synchronization models by unifying time and event based synchronization concepts and o ering a consistent framework in which to handle dynamic media presentation functionality.
Application of Distributed Platforms in a Video Surveillance System
Real-Time Imaging, 2001
O ne of the key aspects in the development of distributed and complex network applications has been the establishment of commonly accepted distributed middleware platforms able to provide an effective solution as far as the development effort versus performance tradeoff is concerned. Based on research work supported on two current key distributed technologies, CORBA and DCOM, the objective of this paper is to show how distributed platforms can be used in a video surveillance system and how they influence the performance of the real-time video service in comparison to other solutions, namely based on Windows Sockets. As far as the performance issue is concerned, the paper identifies the relevant parameters that should be assessed to characterize such video service and presents and discusses results from experiments carried in a testbed. Based on the results obtained, recommendations are formulated concerning the use of distributed platforms in time-bound distributed applications with requirements similar to video surveillance systems.
Related papers
Formalization of the General Video Temporal Synchronization Problem
2010
In this work, we present a theoretical formalization of the temporal synchronization problem and a method to temporally synchronize multiple stationary video cameras with overlapping views of the same scene. The method uses a two stage approach that first approximates the synchronization by tracking moving objects and identifying curvature points. The method then proceeds to refine the estimate using a consensus based matching heuristic to find frames that best agree with the pre-computed camera geometries from stationary background image features. By using the fundamental matrix and the trifocal tensor in the second refinement step, we improve the estimation of the first step and handle a broader more generic range of input scenarios and camera conditions. The method is relatively simple compared to current techniques and is no harder than feature tracking in stage one and computing accurate geometries in stage two. We also provide a robust method to assist synchronization in the presence of inaccurate geometry computation, and a theoretical limit on the accuracy that can be expected from any synchronization system.
Real-time Mosaic for Multi-Camera Videoconferencing
2002
This paper describes a system for high resolution video conferencing. A number of camcorders are used to capture the video, which are then mosaiced to generate a wide angle panoramic view. Furthermore this system is made "real-time" by detecting changes and updating them on the mosaic. This system can be deployed on a single machine or on a cluster for better performance. It is also scalable and shows a good real-time performance. The main application for this system is videoconferencing for distance learning but it can be used for any high resolution broadcasting.
Development of Applications for Real-Time Video Signal Processin
International Conference on Digital Telecommunications (ICDT'06), 2006
The essential purpose of this paper is to describe an architecture, in a simple and complete way, which enables the access and processing of individual pixels corresponding to each frame of a video signal captured in real time; with the aim of facilitating developments made by researchers in the field of image processing, as well as promoting the creation of academic applications in this area.
Scalable software architecture for on-line multi-camera video processing
Real-Time Image and Video Processing 2011, 2011
In this paper we present a scalable software architecture for on-line multi-camera video processing, that guarantees a good trade off between computational power, scalability and flexibility. The software system is modular and its main blocks are the Processing Units (PUs), and the Central Unit. The Central Unit works as a supervisor of the running PUs and each PU manages the acquisition phase and the processing phase. Furthermore, an approach to easily parallelize the desired processing application has been presented. In this paper, as case study, we apply the proposed software architecture to a multi-camera system in order to efficiently manage multiple 2D object detection modules in a real-time scenario. System performance has been evaluated under different load conditions such as number of cameras and image sizes. The results show that the software architecture scales well with the number of camera and can easily works with different image formats respecting the real time constraints. Moreover, the parallelization approach can be used in order to speed up the processing tasks with a low level of overhead.
Frame Rate Exclusive Sync Management of Live Video Streams in Collaborative Mobile Production Environment
We discuss synchronization problem in an emerging type of multimedia applications, called live mobile collaborative video production systems. The mobile character of the production system allows a director to be present at the site where he/she can see the event directly as well as through the mixer display. In such a situation production of a consistent broadcast is sensitive to delay and asynchrony of video streams in the mixer console. In this paper, we propose an algorithm for this situation called "frame rate exclusive sync manager", which draws on existing reactive source control synchronization techniques. It relies solely on frame-rate control and maintains synchronization between live video streams while ensuring minimal delay by dynamically adapting the framerate of the camera feeds based on synchronization offset and network bandwidth health. The algorithm is evaluated by simulation which indicates algorithm's capability of achieving increased synchronization among live streams.
System and software architectures of distributed smart cameras
ACM Transactions on Embedded Computing Systems, 2010
In this article, we describe a distributed, peer-to-peer gesture recognition system along with a software architecture modeling technique and authority control protocol for ubiquitous cameras. This system performs gesture recognition in real time by combining imagery from multiple cameras without using a central server. We propose a system architecture that uses a network of inexpensive cameras to perform in-network video processing. A methodology for transforming well-designed single-node algorithm to distributed system is also proposed. Applications for ubiquitous cameras can be modeled as the composition of a finite-state machine of the system, functional services, and middleware. A service-oriented software architecture is proposed to dynamically reconfigure services when system state changes. By exchanging data and control messages between neighboring sensors, each node can maintain broader view of the environment with integrated video-processing results. Our prototype system i...
A modular software architecture for real-time video processing
Computer Vision Systems, 2001
An Online System for Synchronized Processing of Video and Audio Signals
2006 Canadian Conference on Electrical and Computer Engineering, 2006
For many audiovisual applications, the integration and synchronization of audio and video signals is essential. The objective of this paper is to develop a system that displays the active objects in the captured video signal, integrated with their respective audio signals in the form of text. The video and audio signals are captured and processed separately. The signals are buffered and integrated and synchronized using a time-stamping technique. Time-stamps provide the timing information for each of the audio and video processes, the speech recognition and the object detection, respectively. This information is necessary to correlate the audio packets to the video frames. Hence, integration is achieved without the use of video information, such as lip movements. The results obtained are based on a specific implementation of the speech recognition module, which is determined to be the bottleneck process in the proposed system.
A distributed camera system for multi-resolution surveillance
2009
We describe an architecture for a multi-camera, multi-resolution surveillance system. The aim is to support a set of distributed static and pan-tilt-zoom (PTZ) cameras and visual tracking algorithms, together with a central supervisor unit. Each camera (and possibly pan-tilt device) has a dedicated process and processor. Asynchronous interprocess communications and archiving of data are achieved in a simple and effective way via a central repository, implemented using an SQL database.
An intermedia synchronisation mechanism for multimedia distributed systems
International Journal of Internet Protocol Technology, 2009
Emerging distributed multimedia systems such as Telemedicine, Tele-Conference and IPTV, deal with simultaneous geographically distributed sources by transmitting heterogeneous data, such as text, images, graphics, video and audio. The preservation of temporal relations that consider different data types and simultaneous distributed sources is an open research area. Although several works try to specify and execute at runtime distributed temporal multimedia scenarios, they are far from resolving the problem. This paper proposes a synchronization mechanism to be used at runtime in distributed multimedia systems. One original aspect of the present work is that we avoid the use of a common reference by executing all possible multimedia temporal relations according to their causal dependencies. We emulate the mechanism considering a wide area network environment and using MPEG-4 encoders. The emulation results show that our mechanism is effective in reducing the intermedia synchronization error that can exist. The present work does not require previous knowledge of when, nor for how long, the media involved of a temporal scenario is executed.
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.