Cloud computing has emerged as a solution for large-scale online service applications that requir... more Cloud computing has emerged as a solution for large-scale online service applications that require intense processing, storage and networking capabilities. Cross-cloud communication framework (C3F) was proposed to enable intercommunication among clouds in order to share computing resources. For testing of C3F requires simulation of multiple clouds simultaneously. Although, many cloud simulators are available but multiple cloud simulation under different policies and administrations is not supported. This paper discusses the development of CloudWeb that is a web-based prototype for simulation of multiple clouds concurrently. CloudWeb is developed using open source technologies. Index Term Cloud Computing, Cloud Simulation, Cloud Network, Cloud Communication, Web Services.
In the process of knowledge discovery, the reliability of results depends upon the effectiveness ... more In the process of knowledge discovery, the reliability of results depends upon the effectiveness of attributes selected for decision. The curse of dimensionality refers to the phenomenon in which the excessive number of dimensions affect the analysis. In order to eradicate the curse of dimensionality in text analysis, we are proposing an ontology-based semantic measure for intelligent selection/reduction of features. Among the various text mining techniques, ontology-based mining has a significant contribution to the field. The ontology-based semantic measures, which are mathematical models used to find the similarity between various concepts in the ontology, have made a significant contribution to feature engineering. The proposed measure is an amalgamation of semantic similarity, relatedness, and distance. The measure allows performing an in-depth analysis of various semantic relationships between concepts of the English language. The performance of the measure was evaluated against benchmarked dimension reduction techniques such as PCA. The results show improvement by reducing the size of dimensions up to 35%. The results were further evaluated by training a classifier to validate that the features are not creating any underfit/overfit model. INDEX TERMS Feature engineering, dimension reduction, semantic measures, ontology.
Today's viewers consume video content from a variety of connected devices, including smart phones... more Today's viewers consume video content from a variety of connected devices, including smart phones, tablets, notebooks, TVs, and PCs. This imposes significant challenges for managing video traffic efficiently to ensure an acceptable quality-of-experience (QoE) for the end users as the perceptual quality of video content strongly depends on the properties of the display device and the viewing conditions. State-of-the-art full-reference objective video quality assessment algorithms do not take into account the combined impact of display device properties, viewing conditions, and video resolution while performing video quality assessment. We performed a subjective study in order to understand the impact of aforementioned factors on perceptual video QoE. We also propose a fullreference video QoE measure, named SSIMplus (https://ece.uwaterloo.ca/~z70wang/research/ssimplus/), that provides real-time prediction of the perceptual quality of a video based on human visual system behaviors, video content characteristics (such as spatial and temporal complexity, and video resolution), display device properties (such as screen size, resolution, and brightness), and viewing conditions (such as viewing distance and angle). Experimental results have shown that the proposed algorithm outperforms state-of-the-art video quality measures in terms of accuracy and speed.
Applications of Digital Image Processing XXXVIII, 2015
How to deliver videos to consumers over the network for optimal quality-of-experience (QoE) has b... more How to deliver videos to consumers over the network for optimal quality-of-experience (QoE) has been the central goal of modern video delivery services. Surprisingly, regardless of the large volume of videos being delivered everyday through various systems attempting to improve visual QoE, the actual QoE of end consumers is not properly assessed, not to say using QoE as the key factor in making critical decisions at the video hosting, network and receiving sites. Real-world video streaming systems typically use bitrate as the main video presentation quality indicator, but using the same bitrate to encode different video content could result in drastically different visual QoE, which is further affected by the display device and viewing condition of each individual consumer who receives the video. To correct this, we have to put QoE back to the driver's seat and redesign the video delivery systems. To achieve this goal, a major challenge is to find an objective video presentation QoE predictor that is accurate, fast, easy-to-use, display device adaptive, and provides meaningful QoE predictions across resolution and content. We propose to use the newly developed SSIMplus index (https: //ece.uwaterloo.ca/~z70wang/research/ssimplus/) for this role. We demonstrate that based on SSIMplus, one can develop a smart adaptive video streaming strategy that leads to much smoother visual QoE impossible to achieve using existing adaptive bitrate video streaming approaches. Furthermore, SSIMplus finds many more applications, in live and file-based quality monitoring, in benchmarking video encoders and transcoders, and in guiding network resource allocations.
To achieve optimal video quality under bandwidth and power constraints, modern video coding techn... more To achieve optimal video quality under bandwidth and power constraints, modern video coding techniques employ lossy coding schemes, which often create compression artifacts that may lead to degradation of perceptual video quality. Understanding and quantifying such perceptual artifacts play important roles in the development of effective video compression, streaming and quality enhancement systems. Moreover, the characteristics of compression artifacts evolve over time due to the continuous adoption of novel coding structures and strategies during the development of new video compression standards. In this paper, we reexamine the perceptual artifacts created by standard video compression, summarizing commonly observed spatial and temporal perceptual distortions in compressed video, with emphasis on the perceptual temporal artifacts that have not been well identified or accounted for in previous studies. Furthermore, a floating effect detection method is proposed that not only detects the existence of floating, but also segments the spatial regions where floating occurs * .
IEEE transactions on image processing : a publication of the IEEE Signal Processing Society, Jan 17, 2015
Objective quality assessment of distorted stereoscopic images is a challenging problem, especiall... more Objective quality assessment of distorted stereoscopic images is a challenging problem, especially when the distortions in the left- and right-views are asymmetric. Existing studies suggest that simply averaging the quality of the left- and right-views well predicts the quality of symmetrically distorted stereoscopic images, but generates substantial prediction bias when applied to asymmetrically distorted stereoscopic images. In this study, we first build a database that contains both single-view and symmetrically and asymmetrically distorted stereoscopic images. We then carry out a subjective test, where we find that the quality prediction bias of asymmetrically distorted images could lean towards opposite directions (overestimate or underestimate), depending on the distortion types and levels. Our subjective test also suggests that eye dominance effect does not have strong impact on visual quality decisions of stereoscopic images. Furthermore, we develop an information content an...
2013 Asilomar Conference on Signals, Systems and Computers, 2013
The Structural SIMilarity (SSIM) index has been attracting an increasing amount of attention rece... more The Structural SIMilarity (SSIM) index has been attracting an increasing amount of attention recently in the video coding community as a perceptual criterion for testing and optimizing video codecs. Meanwhile, the arrival of the new MPEG-H/H.265 High Efficiency Video Coding (HEVC) standard creates new opportunities and challenges in perceptual video coding. In this paper, we first elaborate what are the attributes that make SSIM a good candidate for perception-based development of HEVC and future video coding standards for both testing and optimization purposes. We then address the computational issues in practical applications of SSIM in HEVC, in particular the trade-off between efficient computation and accurate estimation of SSIM when working with video codecs that have sophisticated block partitioning structures and aim for encoding videos with a wide range of spatial resolutions.
2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011
The structural similarity (SSIM) index has been found to be a good indicator of perceived image q... more The structural similarity (SSIM) index has been found to be a good indicator of perceived image quality. In this paper, we propose a rate-SSIM optimization scheme for mode selection in H.264/AVC video coding. To derive the Lagrange multiplier based on the properties of input sequences, a novel reduced-reference statistical SSIM model and a source-side information combined rate model are established. The proposed method is fully standard-compatible. Experimental results demonstrate that, compared with conventional rate distortion optimization coding schemes, the proposed scheme can achieve better rate-SSIM performance and provide better visual quality.
2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011
Perceptual image quality assessment (IQA) and sparse signal representation have recently emerged ... more Perceptual image quality assessment (IQA) and sparse signal representation have recently emerged as high-impact research topics in the field of image processing. Here we make one of the first attempts to incorporate the structural similarity (SSIM) index, a promising IQA measure, into the framework of optimal sparse signal representation and approximation. In particular, we introduce a novel image denoising scheme where a modified orthogonal matching pursuit algorithm is proposed for finding the best sparse coefficient vector in maximum-SSIM sense for a given set of linearly independent atoms. Furthermore, a gradient descent algorithm is developed to achieve SSIM-optimal compromise in combining the input and sparse dictionary reconstructed images. Our experimental results show that the proposed method achieves better SSIM performance and provide better visual quality than least square optimal denoising methods.
2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX), 2013
In real-world visual communications, it is a common experience that end-users receive video with ... more In real-world visual communications, it is a common experience that end-users receive video with significantly time-varying quality due to the variations in video content/complexity, codec configuration, and network conditions. How human visual qualityof-experience (QoE) changes with such time-varying video quality is not yet well-understood. To investigate this issue, we conduct subjective experiments designed to examine the quality predictability between individual video segment of relatively constant quality and combined video consisting of multiple segments that have significantly different quality. Our data analysis suggests that simple models that pool segment-level quality, such as linear averaging and weighted-averaging, nonlinear min-and median-filtering, and distortion-weighted averaging, are limited in predicting the overall human quality assessment of the combined video. We thus propose a quality adaptation model that is asymmetrically tuned to increasing and decreasing quality. The proposed asymmetric adaptation (AA) model leads to improved performance of both subjective and objective quality assessment approaches when using segment-level quality scores to predict multi-segment time-varying video quality. The video database together with the subjective data will be made available to the public.
2012 IEEE International Conference on Multimedia and Expo, 2012
Recent advances in video capturing and display technologies, along with the exponentially increas... more Recent advances in video capturing and display technologies, along with the exponentially increasing demand of video services, challenge the video coding research community to design new algorithms able to significantly improve the compression performance of the current H.264/AVC standard. This target is currently gaining evidence with the standardization activities in the High Efficiency Video Coding (HEVC) project. The distortion models used in HEVC are mean squared error (MSE) and sum of absolute difference (SAD). However, they are widely criticized for not correlating well with perceptual image quality. The structural similarity (SSIM) index has been found to be a good indicator of perceived image quality. Meanwhile, it is computationally simple compared with other state-of-theart perceptual quality measures and has a number of desirable mathematical properties for optimization tasks. We propose a perceptual video coding method to improve upon the current HEVC based on an SSIM-inspired divisive normalization scheme as an attempt to transform the DCT domain frame prediction residuals to a perceptually uniform space before encoding. Based on the residual divisive normalization process, we define a distortion model for mode selection and show that such a divisive normalization strategy largely simplifies the subsequent perceptual rate-distortion optimization procedure. We further adjust the divisive normalization factors based on local content of the video frame. Experiments show that the proposed scheme can achieve significant gain in terms of rate-SSIM performance when compared with HEVC.
2011 18th IEEE International Conference on Image Processing, 2011
Perceptually inspired image processing has been an emerging field of study in recent years. Here ... more Perceptually inspired image processing has been an emerging field of study in recent years. Here we make one of the first efforts to incorporate the structural similarity (SSIM) index, a successful perceptual image quality assessment measure, into the framework of non-local means (NLM) image denoising, which is a state-of-the-art method that delivers superior desnoising performance. Specifically, a denoised image patch is obtained by weighted averaging of neighboring patches, where the similarity between patches as well as the weights assigned to the patches are determined based on an estimation of SSIM. A two-stage approach is proposed for robust SSIM estimation in the presence of noise. Moreover, motivated by the ideas behind SSIM, we adjust the contrast and mean of each patch before feeding it into the weighted averaging process. Our experimental results show that the proposed SSIMbased NLM algorithm achieves better SSIM and PSNR performance and provides better visual quality than least square based NLM method.
We propose a perceptual video coding framework based on the divisive normalization scheme, which ... more We propose a perceptual video coding framework based on the divisive normalization scheme, which is found to be an effective approach to model the perceptual sensitivity of biological vision, but has not been fully exploited in the context of video coding. At the macroblock (MB) level, we derive the normalization factors based on the structural similarity (SSIM) index as an attempt to transform the discrete cosine transform domain frame residuals to a perceptually uniform space. We further develop an MB level perceptual mode selection scheme and a frame level global quantization matrix optimization method. Extensive simulations and subjective tests verify that, compared with the H.264/AVC video coding standard, the proposed method can achieve significant gain in terms of rate-SSIM performance and provide better visual quality.
EURASIP Journal on Advances in Signal Processing, 2012
Recently, sparse representation based methods have proven to be successful towards solving image ... more Recently, sparse representation based methods have proven to be successful towards solving image restoration problems. The objective of these methods is to use sparsity prior of the underlying signal in terms of some dictionary and achieve optimal performance in terms of mean-squared error, a metric that has been widely criticized in the literature due to its poor performance as a visual quality predictor. In this work, we make one of the first attempts to employ structural similarity (SSIM) index, a more accurate perceptual image measure, by incorporating it into the framework of sparse signal representation and approximation. Specifically, the proposed optimization problem solves for coefficients with minimum L 0 norm and maximum SSIM index value. Furthermore, a gradient descent algorithm is developed to achieve SSIM-optimal compromise in combining the input and sparse dictionary reconstructed images. We demonstrate the performance of the proposed method by using image denoising and super-resolution methods as examples. Our experimental results show that the proposed SSIM-based sparse representation algorithm achieves better SSIM performance and better visual quality than the corresponding least square-based method.
Significant progress has been made recently towards the next generation video coding standard by ... more Significant progress has been made recently towards the next generation video coding standard by the Joint Collaborative Team on Video Coding (JCT-VC). Recently reported preliminary subjective tests, conducted by JCT-VC members, show that the test model of High Efficiency Video Coding (HEVC) draft codec HM5.0 achieves an average of more than 50% rate savings over H.264 JM18.3 codec without sacrificing subjective quality. Here we study the performance of well-known objective video quality assessment (VQA) models and find that state-of-the-art models, including the Structural Similarity (SSIM), the Multi-Scale SSIM index (MS-SSIM), the Video Quality Metric (VQM), and the MOtion-based Video Integrity Evaluation index (MOVIE), all provide significantly better predictions of subjective video quality than peak signal-to-noise ratio (PSNR). Surprisingly, compared with subjective evaluation scores, all objective VQA models systematically underestimate the coding gain of HEVC-HM5.0 upon H.264-JM18.3. We carried out further subjective tests to study this somewhat unexpected phenomenon by comparing JM18.3 and HM5.0 coded videos in terms of frame-level and sequence-level quality, as well as flickering and ghosting effects. The results provide new insights for the future development of subjective/objective VQA and perceptually-tuned video coding methods.
2010 IEEE International Conference on Image Processing, 2010
The structural similarity (SSIM) index has been shown to be a good perceptual image quality predi... more The structural similarity (SSIM) index has been shown to be a good perceptual image quality predictor. In many real-world applications such as network visual communications, however, SSIM is not applicable because its computation requires full access to the original image. Here we propose a reduced-reference approach that estimates SSIM with only partial information about the original image. Specifically, we extract statistical features from a multi-scale, multi-orientation divisive normalization transform and develop a distortion measure by following the philosophy analogous to that in the construction of SSIM. We found an interesting linear relationship between our reduced-reference SSIM estimate and full-reference SSIM when the image distortion type is fixed. A regression-bydiscretization method is then applied to normalize our measure between image distortion types. We use the LIVE database to test the proposed distortion measure, which shows strong correlations with both SSIM and subjective evaluations. We also demonstrate how our reduced-reference features may be employed to partially repair a distorted image.
Berberis is a wonderful medicinal plant used for various reasons by different communities across ... more Berberis is a wonderful medicinal plant used for various reasons by different communities across traditional landscape of Western Karakoram Mountain Ranges. Since 1950s, its naturally attuned existence faces adverse pressures from bulging human and livestock populations. Out of two subspecies, Berberis pseudumbellata subsp. gilgitica has become critically endangered. Present study was an attempt to assess and explore its uses and their contribution towards its survival and future consequences. A total of 373 people interviewed differentially responded regarding its diverse uses i.e. medicine (92.2%), firewood (19.3%), commercial (2.41%), cultural (2.41%), fodder (16.08%), fencing (19.03%) and grazing (100%). Data was analyzed using Pearson correlational coefficient and student t-test. With the degeneration in ethnobotanical wisdom over generations, its population has also declined. Habitat loss and overgrazing are two major adverse physical factors affecting its population dynamics.
Complex wavelet structural similarity (CW-SSIM) index has been proposed as a promising image simi... more Complex wavelet structural similarity (CW-SSIM) index has been proposed as a promising image similarity measure that is robust to small geometric distortions such as translation, scaling and rotation of images, but how to make the best use of it in image classification problems has not been deeply investigated. In this paper, we propose a novel "feature-extraction free" image classification algorithm based on CW-SSIM and use handwritten digit recognition as an example to demonstrate it. First, a CW-SSIM based unsupervised clustering method is used to divide the training images into clusters and to pick a representative image for each cluster. A supervised learning method based on support vector machines is then employed to maximize the classification accuracy based on CW-SSIM values between an input image and the representative images. Our experiments show that such a conceptually simple image classification method, which does not involve any registration, intensity normalization or sophisticated feature extraction processes, and does not rely on any modeling of the image patterns or distortion processes, achieves competitive performance with reduced computational complexity.
Cloud computing has emerged as a solution for large-scale online service applications that requir... more Cloud computing has emerged as a solution for large-scale online service applications that require intense processing, storage and networking capabilities. Cross-cloud communication framework (C3F) was proposed to enable intercommunication among clouds in order to share computing resources. For testing of C3F requires simulation of multiple clouds simultaneously. Although, many cloud simulators are available but multiple cloud simulation under different policies and administrations is not supported. This paper discusses the development of CloudWeb that is a web-based prototype for simulation of multiple clouds concurrently. CloudWeb is developed using open source technologies. Index Term Cloud Computing, Cloud Simulation, Cloud Network, Cloud Communication, Web Services.
In the process of knowledge discovery, the reliability of results depends upon the effectiveness ... more In the process of knowledge discovery, the reliability of results depends upon the effectiveness of attributes selected for decision. The curse of dimensionality refers to the phenomenon in which the excessive number of dimensions affect the analysis. In order to eradicate the curse of dimensionality in text analysis, we are proposing an ontology-based semantic measure for intelligent selection/reduction of features. Among the various text mining techniques, ontology-based mining has a significant contribution to the field. The ontology-based semantic measures, which are mathematical models used to find the similarity between various concepts in the ontology, have made a significant contribution to feature engineering. The proposed measure is an amalgamation of semantic similarity, relatedness, and distance. The measure allows performing an in-depth analysis of various semantic relationships between concepts of the English language. The performance of the measure was evaluated against benchmarked dimension reduction techniques such as PCA. The results show improvement by reducing the size of dimensions up to 35%. The results were further evaluated by training a classifier to validate that the features are not creating any underfit/overfit model. INDEX TERMS Feature engineering, dimension reduction, semantic measures, ontology.
Today's viewers consume video content from a variety of connected devices, including smart phones... more Today's viewers consume video content from a variety of connected devices, including smart phones, tablets, notebooks, TVs, and PCs. This imposes significant challenges for managing video traffic efficiently to ensure an acceptable quality-of-experience (QoE) for the end users as the perceptual quality of video content strongly depends on the properties of the display device and the viewing conditions. State-of-the-art full-reference objective video quality assessment algorithms do not take into account the combined impact of display device properties, viewing conditions, and video resolution while performing video quality assessment. We performed a subjective study in order to understand the impact of aforementioned factors on perceptual video QoE. We also propose a fullreference video QoE measure, named SSIMplus (https://ece.uwaterloo.ca/~z70wang/research/ssimplus/), that provides real-time prediction of the perceptual quality of a video based on human visual system behaviors, video content characteristics (such as spatial and temporal complexity, and video resolution), display device properties (such as screen size, resolution, and brightness), and viewing conditions (such as viewing distance and angle). Experimental results have shown that the proposed algorithm outperforms state-of-the-art video quality measures in terms of accuracy and speed.
Applications of Digital Image Processing XXXVIII, 2015
How to deliver videos to consumers over the network for optimal quality-of-experience (QoE) has b... more How to deliver videos to consumers over the network for optimal quality-of-experience (QoE) has been the central goal of modern video delivery services. Surprisingly, regardless of the large volume of videos being delivered everyday through various systems attempting to improve visual QoE, the actual QoE of end consumers is not properly assessed, not to say using QoE as the key factor in making critical decisions at the video hosting, network and receiving sites. Real-world video streaming systems typically use bitrate as the main video presentation quality indicator, but using the same bitrate to encode different video content could result in drastically different visual QoE, which is further affected by the display device and viewing condition of each individual consumer who receives the video. To correct this, we have to put QoE back to the driver's seat and redesign the video delivery systems. To achieve this goal, a major challenge is to find an objective video presentation QoE predictor that is accurate, fast, easy-to-use, display device adaptive, and provides meaningful QoE predictions across resolution and content. We propose to use the newly developed SSIMplus index (https: //ece.uwaterloo.ca/~z70wang/research/ssimplus/) for this role. We demonstrate that based on SSIMplus, one can develop a smart adaptive video streaming strategy that leads to much smoother visual QoE impossible to achieve using existing adaptive bitrate video streaming approaches. Furthermore, SSIMplus finds many more applications, in live and file-based quality monitoring, in benchmarking video encoders and transcoders, and in guiding network resource allocations.
To achieve optimal video quality under bandwidth and power constraints, modern video coding techn... more To achieve optimal video quality under bandwidth and power constraints, modern video coding techniques employ lossy coding schemes, which often create compression artifacts that may lead to degradation of perceptual video quality. Understanding and quantifying such perceptual artifacts play important roles in the development of effective video compression, streaming and quality enhancement systems. Moreover, the characteristics of compression artifacts evolve over time due to the continuous adoption of novel coding structures and strategies during the development of new video compression standards. In this paper, we reexamine the perceptual artifacts created by standard video compression, summarizing commonly observed spatial and temporal perceptual distortions in compressed video, with emphasis on the perceptual temporal artifacts that have not been well identified or accounted for in previous studies. Furthermore, a floating effect detection method is proposed that not only detects the existence of floating, but also segments the spatial regions where floating occurs * .
IEEE transactions on image processing : a publication of the IEEE Signal Processing Society, Jan 17, 2015
Objective quality assessment of distorted stereoscopic images is a challenging problem, especiall... more Objective quality assessment of distorted stereoscopic images is a challenging problem, especially when the distortions in the left- and right-views are asymmetric. Existing studies suggest that simply averaging the quality of the left- and right-views well predicts the quality of symmetrically distorted stereoscopic images, but generates substantial prediction bias when applied to asymmetrically distorted stereoscopic images. In this study, we first build a database that contains both single-view and symmetrically and asymmetrically distorted stereoscopic images. We then carry out a subjective test, where we find that the quality prediction bias of asymmetrically distorted images could lean towards opposite directions (overestimate or underestimate), depending on the distortion types and levels. Our subjective test also suggests that eye dominance effect does not have strong impact on visual quality decisions of stereoscopic images. Furthermore, we develop an information content an...
2013 Asilomar Conference on Signals, Systems and Computers, 2013
The Structural SIMilarity (SSIM) index has been attracting an increasing amount of attention rece... more The Structural SIMilarity (SSIM) index has been attracting an increasing amount of attention recently in the video coding community as a perceptual criterion for testing and optimizing video codecs. Meanwhile, the arrival of the new MPEG-H/H.265 High Efficiency Video Coding (HEVC) standard creates new opportunities and challenges in perceptual video coding. In this paper, we first elaborate what are the attributes that make SSIM a good candidate for perception-based development of HEVC and future video coding standards for both testing and optimization purposes. We then address the computational issues in practical applications of SSIM in HEVC, in particular the trade-off between efficient computation and accurate estimation of SSIM when working with video codecs that have sophisticated block partitioning structures and aim for encoding videos with a wide range of spatial resolutions.
2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011
The structural similarity (SSIM) index has been found to be a good indicator of perceived image q... more The structural similarity (SSIM) index has been found to be a good indicator of perceived image quality. In this paper, we propose a rate-SSIM optimization scheme for mode selection in H.264/AVC video coding. To derive the Lagrange multiplier based on the properties of input sequences, a novel reduced-reference statistical SSIM model and a source-side information combined rate model are established. The proposed method is fully standard-compatible. Experimental results demonstrate that, compared with conventional rate distortion optimization coding schemes, the proposed scheme can achieve better rate-SSIM performance and provide better visual quality.
2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011
Perceptual image quality assessment (IQA) and sparse signal representation have recently emerged ... more Perceptual image quality assessment (IQA) and sparse signal representation have recently emerged as high-impact research topics in the field of image processing. Here we make one of the first attempts to incorporate the structural similarity (SSIM) index, a promising IQA measure, into the framework of optimal sparse signal representation and approximation. In particular, we introduce a novel image denoising scheme where a modified orthogonal matching pursuit algorithm is proposed for finding the best sparse coefficient vector in maximum-SSIM sense for a given set of linearly independent atoms. Furthermore, a gradient descent algorithm is developed to achieve SSIM-optimal compromise in combining the input and sparse dictionary reconstructed images. Our experimental results show that the proposed method achieves better SSIM performance and provide better visual quality than least square optimal denoising methods.
2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX), 2013
In real-world visual communications, it is a common experience that end-users receive video with ... more In real-world visual communications, it is a common experience that end-users receive video with significantly time-varying quality due to the variations in video content/complexity, codec configuration, and network conditions. How human visual qualityof-experience (QoE) changes with such time-varying video quality is not yet well-understood. To investigate this issue, we conduct subjective experiments designed to examine the quality predictability between individual video segment of relatively constant quality and combined video consisting of multiple segments that have significantly different quality. Our data analysis suggests that simple models that pool segment-level quality, such as linear averaging and weighted-averaging, nonlinear min-and median-filtering, and distortion-weighted averaging, are limited in predicting the overall human quality assessment of the combined video. We thus propose a quality adaptation model that is asymmetrically tuned to increasing and decreasing quality. The proposed asymmetric adaptation (AA) model leads to improved performance of both subjective and objective quality assessment approaches when using segment-level quality scores to predict multi-segment time-varying video quality. The video database together with the subjective data will be made available to the public.
2012 IEEE International Conference on Multimedia and Expo, 2012
Recent advances in video capturing and display technologies, along with the exponentially increas... more Recent advances in video capturing and display technologies, along with the exponentially increasing demand of video services, challenge the video coding research community to design new algorithms able to significantly improve the compression performance of the current H.264/AVC standard. This target is currently gaining evidence with the standardization activities in the High Efficiency Video Coding (HEVC) project. The distortion models used in HEVC are mean squared error (MSE) and sum of absolute difference (SAD). However, they are widely criticized for not correlating well with perceptual image quality. The structural similarity (SSIM) index has been found to be a good indicator of perceived image quality. Meanwhile, it is computationally simple compared with other state-of-theart perceptual quality measures and has a number of desirable mathematical properties for optimization tasks. We propose a perceptual video coding method to improve upon the current HEVC based on an SSIM-inspired divisive normalization scheme as an attempt to transform the DCT domain frame prediction residuals to a perceptually uniform space before encoding. Based on the residual divisive normalization process, we define a distortion model for mode selection and show that such a divisive normalization strategy largely simplifies the subsequent perceptual rate-distortion optimization procedure. We further adjust the divisive normalization factors based on local content of the video frame. Experiments show that the proposed scheme can achieve significant gain in terms of rate-SSIM performance when compared with HEVC.
2011 18th IEEE International Conference on Image Processing, 2011
Perceptually inspired image processing has been an emerging field of study in recent years. Here ... more Perceptually inspired image processing has been an emerging field of study in recent years. Here we make one of the first efforts to incorporate the structural similarity (SSIM) index, a successful perceptual image quality assessment measure, into the framework of non-local means (NLM) image denoising, which is a state-of-the-art method that delivers superior desnoising performance. Specifically, a denoised image patch is obtained by weighted averaging of neighboring patches, where the similarity between patches as well as the weights assigned to the patches are determined based on an estimation of SSIM. A two-stage approach is proposed for robust SSIM estimation in the presence of noise. Moreover, motivated by the ideas behind SSIM, we adjust the contrast and mean of each patch before feeding it into the weighted averaging process. Our experimental results show that the proposed SSIMbased NLM algorithm achieves better SSIM and PSNR performance and provides better visual quality than least square based NLM method.
We propose a perceptual video coding framework based on the divisive normalization scheme, which ... more We propose a perceptual video coding framework based on the divisive normalization scheme, which is found to be an effective approach to model the perceptual sensitivity of biological vision, but has not been fully exploited in the context of video coding. At the macroblock (MB) level, we derive the normalization factors based on the structural similarity (SSIM) index as an attempt to transform the discrete cosine transform domain frame residuals to a perceptually uniform space. We further develop an MB level perceptual mode selection scheme and a frame level global quantization matrix optimization method. Extensive simulations and subjective tests verify that, compared with the H.264/AVC video coding standard, the proposed method can achieve significant gain in terms of rate-SSIM performance and provide better visual quality.
EURASIP Journal on Advances in Signal Processing, 2012
Recently, sparse representation based methods have proven to be successful towards solving image ... more Recently, sparse representation based methods have proven to be successful towards solving image restoration problems. The objective of these methods is to use sparsity prior of the underlying signal in terms of some dictionary and achieve optimal performance in terms of mean-squared error, a metric that has been widely criticized in the literature due to its poor performance as a visual quality predictor. In this work, we make one of the first attempts to employ structural similarity (SSIM) index, a more accurate perceptual image measure, by incorporating it into the framework of sparse signal representation and approximation. Specifically, the proposed optimization problem solves for coefficients with minimum L 0 norm and maximum SSIM index value. Furthermore, a gradient descent algorithm is developed to achieve SSIM-optimal compromise in combining the input and sparse dictionary reconstructed images. We demonstrate the performance of the proposed method by using image denoising and super-resolution methods as examples. Our experimental results show that the proposed SSIM-based sparse representation algorithm achieves better SSIM performance and better visual quality than the corresponding least square-based method.
Significant progress has been made recently towards the next generation video coding standard by ... more Significant progress has been made recently towards the next generation video coding standard by the Joint Collaborative Team on Video Coding (JCT-VC). Recently reported preliminary subjective tests, conducted by JCT-VC members, show that the test model of High Efficiency Video Coding (HEVC) draft codec HM5.0 achieves an average of more than 50% rate savings over H.264 JM18.3 codec without sacrificing subjective quality. Here we study the performance of well-known objective video quality assessment (VQA) models and find that state-of-the-art models, including the Structural Similarity (SSIM), the Multi-Scale SSIM index (MS-SSIM), the Video Quality Metric (VQM), and the MOtion-based Video Integrity Evaluation index (MOVIE), all provide significantly better predictions of subjective video quality than peak signal-to-noise ratio (PSNR). Surprisingly, compared with subjective evaluation scores, all objective VQA models systematically underestimate the coding gain of HEVC-HM5.0 upon H.264-JM18.3. We carried out further subjective tests to study this somewhat unexpected phenomenon by comparing JM18.3 and HM5.0 coded videos in terms of frame-level and sequence-level quality, as well as flickering and ghosting effects. The results provide new insights for the future development of subjective/objective VQA and perceptually-tuned video coding methods.
2010 IEEE International Conference on Image Processing, 2010
The structural similarity (SSIM) index has been shown to be a good perceptual image quality predi... more The structural similarity (SSIM) index has been shown to be a good perceptual image quality predictor. In many real-world applications such as network visual communications, however, SSIM is not applicable because its computation requires full access to the original image. Here we propose a reduced-reference approach that estimates SSIM with only partial information about the original image. Specifically, we extract statistical features from a multi-scale, multi-orientation divisive normalization transform and develop a distortion measure by following the philosophy analogous to that in the construction of SSIM. We found an interesting linear relationship between our reduced-reference SSIM estimate and full-reference SSIM when the image distortion type is fixed. A regression-bydiscretization method is then applied to normalize our measure between image distortion types. We use the LIVE database to test the proposed distortion measure, which shows strong correlations with both SSIM and subjective evaluations. We also demonstrate how our reduced-reference features may be employed to partially repair a distorted image.
Berberis is a wonderful medicinal plant used for various reasons by different communities across ... more Berberis is a wonderful medicinal plant used for various reasons by different communities across traditional landscape of Western Karakoram Mountain Ranges. Since 1950s, its naturally attuned existence faces adverse pressures from bulging human and livestock populations. Out of two subspecies, Berberis pseudumbellata subsp. gilgitica has become critically endangered. Present study was an attempt to assess and explore its uses and their contribution towards its survival and future consequences. A total of 373 people interviewed differentially responded regarding its diverse uses i.e. medicine (92.2%), firewood (19.3%), commercial (2.41%), cultural (2.41%), fodder (16.08%), fencing (19.03%) and grazing (100%). Data was analyzed using Pearson correlational coefficient and student t-test. With the degeneration in ethnobotanical wisdom over generations, its population has also declined. Habitat loss and overgrazing are two major adverse physical factors affecting its population dynamics.
Complex wavelet structural similarity (CW-SSIM) index has been proposed as a promising image simi... more Complex wavelet structural similarity (CW-SSIM) index has been proposed as a promising image similarity measure that is robust to small geometric distortions such as translation, scaling and rotation of images, but how to make the best use of it in image classification problems has not been deeply investigated. In this paper, we propose a novel "feature-extraction free" image classification algorithm based on CW-SSIM and use handwritten digit recognition as an example to demonstrate it. First, a CW-SSIM based unsupervised clustering method is used to divide the training images into clusters and to pick a representative image for each cluster. A supervised learning method based on support vector machines is then employed to maximize the classification accuracy based on CW-SSIM values between an input image and the representative images. Our experiments show that such a conceptually simple image classification method, which does not involve any registration, intensity normalization or sophisticated feature extraction processes, and does not rely on any modeling of the image patterns or distortion processes, achieves competitive performance with reduced computational complexity.
Uploads
Papers by abdul rehman