In this paper, four subjective video datasets are presented. The considered application is Scalab... more In this paper, four subjective video datasets are presented. The considered application is Scalable Video Coding used as an error-concealment mechanism. The presented datasets explore the relations between encoding parameters and perceived quality, under different network-impairment patterns and involve error-concealment on the decoder's side, to simulate a complete distribution channel. The datasets share a part of common configurations which enables, in the first part of the paper, to compare the outcomes from several Single Stimulus experiments and draw interesting correspondances between different types of distortion. In the second part of the paper, we analyse the performance of three common objective quality metrics on each step of the distribution channel, to identify the possible directions to be followed in order to improve their accuracy in predicting the perceived quality.
We present a framework that allows for simulating a cloud gaming architecture with any DirectX-ba... more We present a framework that allows for simulating a cloud gaming architecture with any DirectX-based video game. The gamer’s screen is captured and compressed as a video stream using the VP8 codec and sent over the network using UDP. We use this framework to evaluate the influence of frame-rate and encoding bit-rate on the Quality of Experience, as perceived by a panel of human test subjects.
Multimedia applications are becoming more and more part of our everyday life. With the fast devel... more Multimedia applications are becoming more and more part of our everyday life. With the fast development of social networks and exchange platforms, users interact with each other through multimedia content, from different places, using different platforms. In order to make these applications available and successful, all the elements involved in the chain between multimedia content production and consumption have to work together with one goal in mind : maximising the Quality of Experience on the user's side. This global issue requires taking into account a high variability of devices, networks, users and consumption scenarios, while addressing technological constraints related to the available resources in terms of network capacities, computation power and storage. In order to gain more knowledge on the provision and assessment of Quality of User Experience, new parameters have to be considered, such as from the psychophysical and environmental context. These efforts are further justified by the emergence of new multimedia concepts and applications, such as 3DTV, Free Viewpoint TV and Mobile TV, which include complex interactions with the users and with their environment.
The QoEMCS workshop has been part of the last two editions of EuroITV. The third edition will once again focus on QoE and multimedia interactions, centered around the evaluation and optimization of interactions between users across different places and platforms.
As smartphones and other mobile multimedia devices are becoming part of our everyday life, suppor... more As smartphones and other mobile multimedia devices are becoming part of our everyday life, supported by the continuous development of mobile broadband services all around the world, mobile TV is naturally becoming a more and more popular application. Mainly because of the high cost of content production and delivery, most of current mobile TV services provide only content originally designed for more traditional applications such as residential TV. However, the consumption scenario is drastically different, which calls for a redefinition of both the content providing chain and the quality evaluation procedures. In this paper, we present a new video database especially designed for mobile TV quality assessment. After reporting the results of a broad study to provide content creation and editing guidelines adapted for mobile TV, we analyze the impact of these guidelines on the perception of quality through a subjective experiment.
In multimedia quality assessment, observer ratings are typically averaged into mean opinion score... more In multimedia quality assessment, observer ratings are typically averaged into mean opinion scores (MOS) to obtain a subjective ground truth for a set of stimuli. Valuable information about individual observer rating behaviour and inter-observer differences is lost during this process. Such information, however, can be useful to improve subjective experiment procedures and quality of experience prediction models. In this paper, we therefore present an inter-observer analysis framework that tackles the quality assessment problem from an inverse angle, setting the focus on individual observer differences rather than stimuli differences. The framework consists of a set of procedures for inter-observer analysis as well as the necessary considerations during pre-processing and post-processing. The aim of this paper is to raise awareness that sole consideration of MOS simplifies quality assessment too much, especially given the ever increasing complexity of multimedia quality assessment.
User-generated videos are becoming a very popular way for capturing and sharing material. The cre... more User-generated videos are becoming a very popular way for capturing and sharing material. The creation and consumption of these videos is quite different from the traditional professionally designed videos. Consumer-range cameras are often used to generate the videos, in uncontrolled conditions, possibly operated without the same considerations regarding camera stabilization or lighting conditions. If the videos are then shared on a video sharing platform, they often have to go through a re-encoding step that is likely to degrade their original quality. At last, if consumers are watching the videos on a mobile phone, the viewing conditions might have an impact on the perception of quality of the videos. In this paper, we evaluate the whole chain from production to consumption of user-generated videos through a subjective quality assessment experiment conducted on 29 naive viewers. The results of the experiment allow us to identify camera shaking and re-encoding as two main factors of variation of the video quality in this context. However, the influence of the viewing context can not be considered significant, even if the results show that the videos degradations are perceived less severely in a distracting environment.
A popular and effective method to protect professional content against illegal distribution is to... more A popular and effective method to protect professional content against illegal distribution is to use watermarking, which inserts a robust and imperceptible fingerprint in the data using complex information hiding techniques. Video DNA watermarking is a state-of-the-art technique which enables shifting of the computational complexity to the server side. Despite the high level of protection it provides, this technique comes at a price in terms of storage space, as several versions of the same content have to be stored on the server. It is possible to limit the impact in terms of bitrate by slightly decreasing the quality of the final video. In this paper, we evaluate the impact of using Video DNA watermarking on the perceived quality by presenting the results of a standardized subjective experiment. We study various strategies to minimize the impact on the quality while maintaining the same overall bitrate, acting on the spatial resolution of the videos and bitrate repartition schemes.
Subjective video quality experiments are usually conducted inside controlled lab environments, wh... more Subjective video quality experiments are usually conducted inside controlled lab environments, where stringent demands are imposed in terms of lighting conditions, screen calibration and position of the test subjects. However, these controlled lab environments might differ from a more natural setting such as watching television at home. This, in turn, will influence subjects’ Quality of Experience. In previous research, we conducted a series of subjective experiments, mimicking the natural environment of the targeted use cases as much as possible. In this paper, we provide an overview of these conducted studies and summarize our main research findings. Our results show that the environment and overall experimental setup influence the primary focus of the test subjects. Consequently, a significant difference in impairment visibility, tolerance and annoyance is observed compared to conducting experiments in pristine lab environments. Our results highlight the importance of considering the targeted use case and assessing video quality in more natural environments.
This paper considers the problem of decreasing the number of video sequences to present in a subj... more This paper considers the problem of decreasing the number of video sequences to present in a subjective experiment so that the duration stays under the standard recommendations, while allowing to evaluate the perceived quality on a large set of configurations and contents. We present the constraints and related issues, then compare three selection algorithms through a series of subjective experiments. We analyse the correlation between the votes given to a set of videos in an initial large-scale test, and the votes given to three subsets of these videos in separate smaller experiments.
In a mobile environment, many distractions can draw the attention of the user away from his phone... more In a mobile environment, many distractions can draw the attention of the user away from his phone, even if he is watching video content. The power consumed by the different elements involved in the playback of the video when the user is not watching can then be considered as not optimally used. In this paper we present an interactive video player allowing to temporarily interrupt the playback when the user is looking away, and resume it when the user starts looking at the phone again. In order to detect user inattention, we exploit as a face detection module using the front-facing camera embedded in most current smartphones. We analyze the power consumption of the phone when running our interactive video player in controlled and realistic scenarios, and study the energy savings induced by the attention-aware playback interruption. Additionally, we study the influence of video bitrate and content on power consumption. Finally, we propose a cloud-based approach in which the image captured by the front camera is sent to a server, which performs the face detection. We show that the power savings are significant using this approach, and can compensate the consumption added by the increased wireless traffic.
Degradation of network performance during video transmission may lead to disturbing visual artifa... more Degradation of network performance during video transmission may lead to disturbing visual artifacts. Some packets might be lost, corrupted or delayed, making it impossible to properly decode the video data on time at the receiver. The quality of the error-concealment technique, as well as the spatial and temporal position of the artifacts have a large impact on the perceived quality after decoding. In this paper, we use the spatial scalability feature of Scalable Video Coding (SVC) for error-concealment. This enables the transmission of a lower resolution video with a higher robustness, for example using unequal error protection. Under the assumption that only the higher resolution video would be affected, we evaluated the visual impact of packet losses in a large scale subjective video quality experiment using the Absolute Category Rating method. The number of impairments, the duration, and the interval between impairments as well the quality of the encoded lower resolution video are varied in a systematic evaluation. This allows for analyzing the influence of each factor both independently and jointly.
In video coding, it is commonly accepted that the encoding parameters such as the quantization st... more In video coding, it is commonly accepted that the encoding parameters such as the quantization step-size have an influence on the perceived quality. It is also sometimes accepted that using given encoding parameters, the perceived quality does not change significantly according to the encoded source content. In this paper, we present the outcomes of two video subjective quality assessment experiments in the context of Scalable Video Coding. We encoded a large set of video sequences under a group of constant quality scenarios based on two spatially scalable layers. One first experiment explores of the relation between a wide range of quantization parameters for each layer and the perceived quality, while the second experiment uses a subset of the encoding scenarios on a large number of video sequences. The two experiments are aligned on a common scale using a set of shared processed video sequences, resulting in a database containing the subjective scores for 60 different sources combined with 20 SVC scenarios. We propose a detailed analysis of the experimental results of the two experiments, bringing a clear insight of the relation between the encoding parameters combination of the scalable layers and the perceived quality, as well as spreading light on the differences in terms of quality depending on the encoded source content. As an endeavour to analyse these differences, we propose a classification of the sources with regards to their relative behaviour when compared to the average of other source contents. We use this classification to identify potential factors to explain the differences between source contents.
Scalable Video Coding (SVC) provides a way to encapsulate several video layers with increasing qu... more Scalable Video Coding (SVC) provides a way to encapsulate several video layers with increasing quality and resolution in a single bitstream. Thus it is particularly adapted to address heterogeneous networks and a wide variety of decoding devices. In this paper, we evaluate the interest of SVC in a different context, which is error concealment after transmission on networks subject to packet loss. The encoded scalable video streams contain two layers with different spatial and temporal resolutions designed for mobile video communications with medium size and average to low bitrates. The main idea is to use the base layer to conceal errors in the higher layers if they are corrupted or lost. The base layer is first upscaled either spatially or temporally to reach the same resolution as the layer to conceal. Two error-concealment techniques using the base layer are then proposed for the MPEG-4 SVC standard, involving frame-level concealment and pixel-level concealment. These techniques are compared to the upscaled base layer as well as to a classical single-layer MPEG-4 AVC/H.264 error-concealment technique. The comparison is carried out through a subjective experiment, in order to evaluate the Quality-of-Experience of the proposed techniques. We study several scenarios involving various bitrates and resolutions for the base layer of the SVC streams. The results show that SVC-based error concealment can provide significantly higher visual quality than single-layer-based techniques. Moreover, we demonstrate that the resolution and bitrate of the base layer have a strong impact on the perceived quality of the concealment.
In this paper we use a common set between three subjective tests to build a linear mapping of the... more In this paper we use a common set between three subjective tests to build a linear mapping of the results of two tests onto the scale of one test identified as the reference test. We present our low-cost approach for the design of the common set and discuss the choice of the reference test. The mapping is then used to merge the outcomes of the three tests and provide an interesting comparison of the impact of coding artifacts, transmission errors and error-concealment in the context of Scalable Video Coding.
The upcoming High-Definition format for video display provides high-quality content, especially w... more The upcoming High-Definition format for video display provides high-quality content, especially when displayed on adapted devices. When combined with video coding techniques such as MPEG-4 AVC/H.264, the transmission of High-Definition video content on broadcast networks becomes possible. Nonetheless, transmitting and decoding such video content is a real challenge. Therefore, intermediate formats based on lower frame resolutions or interlaced coding are still provided to address targets with limited resources. Using these formats, the final video quality depends on the postprocessing tools employed at the receiver to up-sample and de-interlace these streams. In this paper, we compare the full-HD format to three possible scenarios to generate a full-HD stream from intermediate formats. We present the results of subjective tests that compare the visual quality of each scenario when using the same bitrate. The results show that using the same bitrate, the videos generated from lower-resolution formats reach similar quality compared to the full-HD videos.
Scalable Video Coding provides several levels of video encapsulated in a single video stream. In ... more Scalable Video Coding provides several levels of video encapsulated in a single video stream. In a transmission scenario such as broadcasting, this structure is quite advantageous as it can be used to address heterogeneous decoding targets with variable needs and requirements. However, this adaptability comes at a slight cost in coding efficiency when compared to single-layer coding. Based on subjective experiments, this cost is evaluated in this paper by comparing the new MPEG-4 Scalable Video Coding (SVC) standard with the now-established MPEG-4 AVC standard. Two scenarios are analyzed in the context of mobile transmission applications. The first scenario uses the same bitrate for SVC and AVC, leading to a slightly lower PSNR for SVC. The second scenario uses the same PSNR for SVC and AVC, leading to a slightly lower bitrate for AVC. The results of the subjective tests illustrate several interesting aspects of the relation between the performance of the two standards. First, we observe that the offset between AVC and SVC is not severe, though statistically significant in terms of user Mean Opinion Score (MOS) in such a context. Second, while adding another layer to SVC always leads to a performance loss, the impact of this loss decreases when the number of layers increases.
The aim of this paper is to give an overview of the SVC4QoE project which purpose is to use Scala... more The aim of this paper is to give an overview of the SVC4QoE project which purpose is to use Scalable Video Coding (SVC) to optimize video transmission in terms of Quality of Experience over DVB-T2 channels. The originality of this project is to consider the influence of the whole chain of treatments performed on the video data in terms of user-perceived quality. The encoding process, as well as transmission, decoding and display are included in the optimization process. Particularly, the multi-layer structure of SVC is to be exploited to circumvent alterations of the video related to transmission on error-prone networks. Combining SVC with QoE thus provides a way to reduce network operating costs, while increasing the quality from the user's point of view. Several innovations are also to be mentioned, such as the use of a real-time open-source SVC decoder, evaluation of visual quality through subjective quality assessment tests, transmission and synchronization of SVC layers on the receiver's side, and development of a DVB-T2 Gateway which implements the multi-PLP functionality in an SVC context.
This paper presents an attractive rate control scheme for the new MPEG-4 Scalable Video Coding st... more This paper presents an attractive rate control scheme for the new MPEG-4 Scalable Video Coding standard. Our scheme enables us to control the bitrate at the output of the encoder on each video layer with great accuracy. Each frame is encoded only once, so that the computational complexity of the whole scheme is very low. The three spatial, temporal and quality scalabilities are handled correctly, as well as inter layer prediction and hierarchical B frames. A linear bitrate model is used to predict the output bitrate for a frame, based on a simple and effective framework called \boldmath{$\rho$}-domain. A coding-complexity measure is also introduced to dispatch the available bits among the frames, in order to reach a constant quality throughout the encoded video stream. To attest the performances of our rate control scheme, we present comprehensive results on some representative scalable video set-ups.
Rate control is a capital issue in video coding. It allows a regulation of the bitrate out from t... more Rate control is a capital issue in video coding. It allows a regulation of the bitrate out from the encoder, to cope with some network transmission or quality constraints. Scalable Video Coding emerged several years ago as an answer to the growing need of application-adaptable video streams. Although the interest of scalable video coding has been confirmed by recent studies, it can not be used in practical contexts without proper rate control techniques. In this paper we present a new rate control scheme for scalable video, based on a simple yet attractive bitrate modeling framework called $\rho$-domain. Our scheme performs accurate rate control on spatial, temporal and quality scalabilities, while maintaining a constant PSNR. Inter layer prediction is also handled effectively.
Scalable Video Coding was designed in response to the growing need for flexibility in video trans... more Scalable Video Coding was designed in response to the growing need for flexibility in video transmission over networks and channels. MPEG-4 Scalable Video Coding (SVC) is a recently finalized standard which introduces new coding tools such as spatial, temporal and quality scalability, to produce a layer-based scalable video stream. Additionally, inter-layer prediction allows a layer to use information from other layers as a basis for motion and texture prediction, improving the overall coding efficiency. Rate control is a capital issue in video coding, as it is designed to regulate the bitrate at the output of the encoder and keep it close to a specified constraint. Whereas rate control has been extensively studied for non-scalable video coding, only few propositions were made for scalable video coding. In this paper, we adapt an attractive rate control approach, based on a bitrate modeling framework called $\rho$-domain, for scalable video coding. We show that this model performs well on all spatial, temporal and quality scalabilities, and handles inter-layer prediction quite accurately. After validating the approach in MPEG-4 SVC, we use the $\rho$-domain model to build a simple-accurate rate control scheme. Results show that the mean frame bitrate error is below 7\% on a representative set of configurations, while the impact on the complexity of the encoder is very low.
In this paper, four subjective video datasets are presented. The considered application is Scalab... more In this paper, four subjective video datasets are presented. The considered application is Scalable Video Coding used as an error-concealment mechanism. The presented datasets explore the relations between encoding parameters and perceived quality, under different network-impairment patterns and involve error-concealment on the decoder's side, to simulate a complete distribution channel. The datasets share a part of common configurations which enables, in the first part of the paper, to compare the outcomes from several Single Stimulus experiments and draw interesting correspondances between different types of distortion. In the second part of the paper, we analyse the performance of three common objective quality metrics on each step of the distribution channel, to identify the possible directions to be followed in order to improve their accuracy in predicting the perceived quality.
We present a framework that allows for simulating a cloud gaming architecture with any DirectX-ba... more We present a framework that allows for simulating a cloud gaming architecture with any DirectX-based video game. The gamer’s screen is captured and compressed as a video stream using the VP8 codec and sent over the network using UDP. We use this framework to evaluate the influence of frame-rate and encoding bit-rate on the Quality of Experience, as perceived by a panel of human test subjects.
Multimedia applications are becoming more and more part of our everyday life. With the fast devel... more Multimedia applications are becoming more and more part of our everyday life. With the fast development of social networks and exchange platforms, users interact with each other through multimedia content, from different places, using different platforms. In order to make these applications available and successful, all the elements involved in the chain between multimedia content production and consumption have to work together with one goal in mind : maximising the Quality of Experience on the user's side. This global issue requires taking into account a high variability of devices, networks, users and consumption scenarios, while addressing technological constraints related to the available resources in terms of network capacities, computation power and storage. In order to gain more knowledge on the provision and assessment of Quality of User Experience, new parameters have to be considered, such as from the psychophysical and environmental context. These efforts are further justified by the emergence of new multimedia concepts and applications, such as 3DTV, Free Viewpoint TV and Mobile TV, which include complex interactions with the users and with their environment.
The QoEMCS workshop has been part of the last two editions of EuroITV. The third edition will once again focus on QoE and multimedia interactions, centered around the evaluation and optimization of interactions between users across different places and platforms.
As smartphones and other mobile multimedia devices are becoming part of our everyday life, suppor... more As smartphones and other mobile multimedia devices are becoming part of our everyday life, supported by the continuous development of mobile broadband services all around the world, mobile TV is naturally becoming a more and more popular application. Mainly because of the high cost of content production and delivery, most of current mobile TV services provide only content originally designed for more traditional applications such as residential TV. However, the consumption scenario is drastically different, which calls for a redefinition of both the content providing chain and the quality evaluation procedures. In this paper, we present a new video database especially designed for mobile TV quality assessment. After reporting the results of a broad study to provide content creation and editing guidelines adapted for mobile TV, we analyze the impact of these guidelines on the perception of quality through a subjective experiment.
In multimedia quality assessment, observer ratings are typically averaged into mean opinion score... more In multimedia quality assessment, observer ratings are typically averaged into mean opinion scores (MOS) to obtain a subjective ground truth for a set of stimuli. Valuable information about individual observer rating behaviour and inter-observer differences is lost during this process. Such information, however, can be useful to improve subjective experiment procedures and quality of experience prediction models. In this paper, we therefore present an inter-observer analysis framework that tackles the quality assessment problem from an inverse angle, setting the focus on individual observer differences rather than stimuli differences. The framework consists of a set of procedures for inter-observer analysis as well as the necessary considerations during pre-processing and post-processing. The aim of this paper is to raise awareness that sole consideration of MOS simplifies quality assessment too much, especially given the ever increasing complexity of multimedia quality assessment.
User-generated videos are becoming a very popular way for capturing and sharing material. The cre... more User-generated videos are becoming a very popular way for capturing and sharing material. The creation and consumption of these videos is quite different from the traditional professionally designed videos. Consumer-range cameras are often used to generate the videos, in uncontrolled conditions, possibly operated without the same considerations regarding camera stabilization or lighting conditions. If the videos are then shared on a video sharing platform, they often have to go through a re-encoding step that is likely to degrade their original quality. At last, if consumers are watching the videos on a mobile phone, the viewing conditions might have an impact on the perception of quality of the videos. In this paper, we evaluate the whole chain from production to consumption of user-generated videos through a subjective quality assessment experiment conducted on 29 naive viewers. The results of the experiment allow us to identify camera shaking and re-encoding as two main factors of variation of the video quality in this context. However, the influence of the viewing context can not be considered significant, even if the results show that the videos degradations are perceived less severely in a distracting environment.
A popular and effective method to protect professional content against illegal distribution is to... more A popular and effective method to protect professional content against illegal distribution is to use watermarking, which inserts a robust and imperceptible fingerprint in the data using complex information hiding techniques. Video DNA watermarking is a state-of-the-art technique which enables shifting of the computational complexity to the server side. Despite the high level of protection it provides, this technique comes at a price in terms of storage space, as several versions of the same content have to be stored on the server. It is possible to limit the impact in terms of bitrate by slightly decreasing the quality of the final video. In this paper, we evaluate the impact of using Video DNA watermarking on the perceived quality by presenting the results of a standardized subjective experiment. We study various strategies to minimize the impact on the quality while maintaining the same overall bitrate, acting on the spatial resolution of the videos and bitrate repartition schemes.
Subjective video quality experiments are usually conducted inside controlled lab environments, wh... more Subjective video quality experiments are usually conducted inside controlled lab environments, where stringent demands are imposed in terms of lighting conditions, screen calibration and position of the test subjects. However, these controlled lab environments might differ from a more natural setting such as watching television at home. This, in turn, will influence subjects’ Quality of Experience. In previous research, we conducted a series of subjective experiments, mimicking the natural environment of the targeted use cases as much as possible. In this paper, we provide an overview of these conducted studies and summarize our main research findings. Our results show that the environment and overall experimental setup influence the primary focus of the test subjects. Consequently, a significant difference in impairment visibility, tolerance and annoyance is observed compared to conducting experiments in pristine lab environments. Our results highlight the importance of considering the targeted use case and assessing video quality in more natural environments.
This paper considers the problem of decreasing the number of video sequences to present in a subj... more This paper considers the problem of decreasing the number of video sequences to present in a subjective experiment so that the duration stays under the standard recommendations, while allowing to evaluate the perceived quality on a large set of configurations and contents. We present the constraints and related issues, then compare three selection algorithms through a series of subjective experiments. We analyse the correlation between the votes given to a set of videos in an initial large-scale test, and the votes given to three subsets of these videos in separate smaller experiments.
In a mobile environment, many distractions can draw the attention of the user away from his phone... more In a mobile environment, many distractions can draw the attention of the user away from his phone, even if he is watching video content. The power consumed by the different elements involved in the playback of the video when the user is not watching can then be considered as not optimally used. In this paper we present an interactive video player allowing to temporarily interrupt the playback when the user is looking away, and resume it when the user starts looking at the phone again. In order to detect user inattention, we exploit as a face detection module using the front-facing camera embedded in most current smartphones. We analyze the power consumption of the phone when running our interactive video player in controlled and realistic scenarios, and study the energy savings induced by the attention-aware playback interruption. Additionally, we study the influence of video bitrate and content on power consumption. Finally, we propose a cloud-based approach in which the image captured by the front camera is sent to a server, which performs the face detection. We show that the power savings are significant using this approach, and can compensate the consumption added by the increased wireless traffic.
Degradation of network performance during video transmission may lead to disturbing visual artifa... more Degradation of network performance during video transmission may lead to disturbing visual artifacts. Some packets might be lost, corrupted or delayed, making it impossible to properly decode the video data on time at the receiver. The quality of the error-concealment technique, as well as the spatial and temporal position of the artifacts have a large impact on the perceived quality after decoding. In this paper, we use the spatial scalability feature of Scalable Video Coding (SVC) for error-concealment. This enables the transmission of a lower resolution video with a higher robustness, for example using unequal error protection. Under the assumption that only the higher resolution video would be affected, we evaluated the visual impact of packet losses in a large scale subjective video quality experiment using the Absolute Category Rating method. The number of impairments, the duration, and the interval between impairments as well the quality of the encoded lower resolution video are varied in a systematic evaluation. This allows for analyzing the influence of each factor both independently and jointly.
In video coding, it is commonly accepted that the encoding parameters such as the quantization st... more In video coding, it is commonly accepted that the encoding parameters such as the quantization step-size have an influence on the perceived quality. It is also sometimes accepted that using given encoding parameters, the perceived quality does not change significantly according to the encoded source content. In this paper, we present the outcomes of two video subjective quality assessment experiments in the context of Scalable Video Coding. We encoded a large set of video sequences under a group of constant quality scenarios based on two spatially scalable layers. One first experiment explores of the relation between a wide range of quantization parameters for each layer and the perceived quality, while the second experiment uses a subset of the encoding scenarios on a large number of video sequences. The two experiments are aligned on a common scale using a set of shared processed video sequences, resulting in a database containing the subjective scores for 60 different sources combined with 20 SVC scenarios. We propose a detailed analysis of the experimental results of the two experiments, bringing a clear insight of the relation between the encoding parameters combination of the scalable layers and the perceived quality, as well as spreading light on the differences in terms of quality depending on the encoded source content. As an endeavour to analyse these differences, we propose a classification of the sources with regards to their relative behaviour when compared to the average of other source contents. We use this classification to identify potential factors to explain the differences between source contents.
Scalable Video Coding (SVC) provides a way to encapsulate several video layers with increasing qu... more Scalable Video Coding (SVC) provides a way to encapsulate several video layers with increasing quality and resolution in a single bitstream. Thus it is particularly adapted to address heterogeneous networks and a wide variety of decoding devices. In this paper, we evaluate the interest of SVC in a different context, which is error concealment after transmission on networks subject to packet loss. The encoded scalable video streams contain two layers with different spatial and temporal resolutions designed for mobile video communications with medium size and average to low bitrates. The main idea is to use the base layer to conceal errors in the higher layers if they are corrupted or lost. The base layer is first upscaled either spatially or temporally to reach the same resolution as the layer to conceal. Two error-concealment techniques using the base layer are then proposed for the MPEG-4 SVC standard, involving frame-level concealment and pixel-level concealment. These techniques are compared to the upscaled base layer as well as to a classical single-layer MPEG-4 AVC/H.264 error-concealment technique. The comparison is carried out through a subjective experiment, in order to evaluate the Quality-of-Experience of the proposed techniques. We study several scenarios involving various bitrates and resolutions for the base layer of the SVC streams. The results show that SVC-based error concealment can provide significantly higher visual quality than single-layer-based techniques. Moreover, we demonstrate that the resolution and bitrate of the base layer have a strong impact on the perceived quality of the concealment.
In this paper we use a common set between three subjective tests to build a linear mapping of the... more In this paper we use a common set between three subjective tests to build a linear mapping of the results of two tests onto the scale of one test identified as the reference test. We present our low-cost approach for the design of the common set and discuss the choice of the reference test. The mapping is then used to merge the outcomes of the three tests and provide an interesting comparison of the impact of coding artifacts, transmission errors and error-concealment in the context of Scalable Video Coding.
The upcoming High-Definition format for video display provides high-quality content, especially w... more The upcoming High-Definition format for video display provides high-quality content, especially when displayed on adapted devices. When combined with video coding techniques such as MPEG-4 AVC/H.264, the transmission of High-Definition video content on broadcast networks becomes possible. Nonetheless, transmitting and decoding such video content is a real challenge. Therefore, intermediate formats based on lower frame resolutions or interlaced coding are still provided to address targets with limited resources. Using these formats, the final video quality depends on the postprocessing tools employed at the receiver to up-sample and de-interlace these streams. In this paper, we compare the full-HD format to three possible scenarios to generate a full-HD stream from intermediate formats. We present the results of subjective tests that compare the visual quality of each scenario when using the same bitrate. The results show that using the same bitrate, the videos generated from lower-resolution formats reach similar quality compared to the full-HD videos.
Scalable Video Coding provides several levels of video encapsulated in a single video stream. In ... more Scalable Video Coding provides several levels of video encapsulated in a single video stream. In a transmission scenario such as broadcasting, this structure is quite advantageous as it can be used to address heterogeneous decoding targets with variable needs and requirements. However, this adaptability comes at a slight cost in coding efficiency when compared to single-layer coding. Based on subjective experiments, this cost is evaluated in this paper by comparing the new MPEG-4 Scalable Video Coding (SVC) standard with the now-established MPEG-4 AVC standard. Two scenarios are analyzed in the context of mobile transmission applications. The first scenario uses the same bitrate for SVC and AVC, leading to a slightly lower PSNR for SVC. The second scenario uses the same PSNR for SVC and AVC, leading to a slightly lower bitrate for AVC. The results of the subjective tests illustrate several interesting aspects of the relation between the performance of the two standards. First, we observe that the offset between AVC and SVC is not severe, though statistically significant in terms of user Mean Opinion Score (MOS) in such a context. Second, while adding another layer to SVC always leads to a performance loss, the impact of this loss decreases when the number of layers increases.
The aim of this paper is to give an overview of the SVC4QoE project which purpose is to use Scala... more The aim of this paper is to give an overview of the SVC4QoE project which purpose is to use Scalable Video Coding (SVC) to optimize video transmission in terms of Quality of Experience over DVB-T2 channels. The originality of this project is to consider the influence of the whole chain of treatments performed on the video data in terms of user-perceived quality. The encoding process, as well as transmission, decoding and display are included in the optimization process. Particularly, the multi-layer structure of SVC is to be exploited to circumvent alterations of the video related to transmission on error-prone networks. Combining SVC with QoE thus provides a way to reduce network operating costs, while increasing the quality from the user's point of view. Several innovations are also to be mentioned, such as the use of a real-time open-source SVC decoder, evaluation of visual quality through subjective quality assessment tests, transmission and synchronization of SVC layers on the receiver's side, and development of a DVB-T2 Gateway which implements the multi-PLP functionality in an SVC context.
This paper presents an attractive rate control scheme for the new MPEG-4 Scalable Video Coding st... more This paper presents an attractive rate control scheme for the new MPEG-4 Scalable Video Coding standard. Our scheme enables us to control the bitrate at the output of the encoder on each video layer with great accuracy. Each frame is encoded only once, so that the computational complexity of the whole scheme is very low. The three spatial, temporal and quality scalabilities are handled correctly, as well as inter layer prediction and hierarchical B frames. A linear bitrate model is used to predict the output bitrate for a frame, based on a simple and effective framework called \boldmath{$\rho$}-domain. A coding-complexity measure is also introduced to dispatch the available bits among the frames, in order to reach a constant quality throughout the encoded video stream. To attest the performances of our rate control scheme, we present comprehensive results on some representative scalable video set-ups.
Rate control is a capital issue in video coding. It allows a regulation of the bitrate out from t... more Rate control is a capital issue in video coding. It allows a regulation of the bitrate out from the encoder, to cope with some network transmission or quality constraints. Scalable Video Coding emerged several years ago as an answer to the growing need of application-adaptable video streams. Although the interest of scalable video coding has been confirmed by recent studies, it can not be used in practical contexts without proper rate control techniques. In this paper we present a new rate control scheme for scalable video, based on a simple yet attractive bitrate modeling framework called $\rho$-domain. Our scheme performs accurate rate control on spatial, temporal and quality scalabilities, while maintaining a constant PSNR. Inter layer prediction is also handled effectively.
Scalable Video Coding was designed in response to the growing need for flexibility in video trans... more Scalable Video Coding was designed in response to the growing need for flexibility in video transmission over networks and channels. MPEG-4 Scalable Video Coding (SVC) is a recently finalized standard which introduces new coding tools such as spatial, temporal and quality scalability, to produce a layer-based scalable video stream. Additionally, inter-layer prediction allows a layer to use information from other layers as a basis for motion and texture prediction, improving the overall coding efficiency. Rate control is a capital issue in video coding, as it is designed to regulate the bitrate at the output of the encoder and keep it close to a specified constraint. Whereas rate control has been extensively studied for non-scalable video coding, only few propositions were made for scalable video coding. In this paper, we adapt an attractive rate control approach, based on a bitrate modeling framework called $\rho$-domain, for scalable video coding. We show that this model performs well on all spatial, temporal and quality scalabilities, and handles inter-layer prediction quite accurately. After validating the approach in MPEG-4 SVC, we use the $\rho$-domain model to build a simple-accurate rate control scheme. Results show that the mean frame bitrate error is below 7\% on a representative set of configurations, while the impact on the complexity of the encoder is very low.
Uploads
Papers by Yohann Pitrey
The QoEMCS workshop has been part of the last two editions of EuroITV. The third edition will once again focus on QoE and multimedia interactions, centered around the evaluation and optimization of interactions between users across different places and platforms.
differences rather than stimuli differences. The framework consists of a set of procedures for inter-observer analysis as well as the necessary considerations during pre-processing and post-processing. The aim of this paper is to raise awareness that sole consideration of MOS simplifies quality assessment too much, especially given the ever increasing complexity of multimedia quality assessment.
The QoEMCS workshop has been part of the last two editions of EuroITV. The third edition will once again focus on QoE and multimedia interactions, centered around the evaluation and optimization of interactions between users across different places and platforms.
differences rather than stimuli differences. The framework consists of a set of procedures for inter-observer analysis as well as the necessary considerations during pre-processing and post-processing. The aim of this paper is to raise awareness that sole consideration of MOS simplifies quality assessment too much, especially given the ever increasing complexity of multimedia quality assessment.