The brightness (perceived intensity) of a region of visual space depends on its luminance and on ... more The brightness (perceived intensity) of a region of visual space depends on its luminance and on the luminance of nearby regions. This phenomenon is called brightness induction and includes both brightness contrast and assimilation. Historically, and on a purely descriptive level, brightness contrast refers to a directional shift in target brightness away from the brightness of an adjacent region while assimilation refers to a brightness shift toward that of an adjacent region. In order to understand mechanisms, it is important to differentiate the descriptive terms contrast and assimilation from the optical and/or neural processes, often similarly named, which cause the effects. Experiment 1 isolated the effect on target patch (64 cd/m2) matching luminance (brightness) of six surround-ring widths (0.1°–24.5°) varied over 11 surround-ring luminances (32–96 cd/m2). Using the same observers, Experiment 2 examined the effect of the identical surround-ring parameters on target patch mat...
Two supra-threshold lateral interactions, the grating-induction effect,1 and the contrast–contras... more Two supra-threshold lateral interactions, the grating-induction effect,1 and the contrast–contrast effect,2,3 were compared regarding their dependence upon inducing grating spatial frequency, inducing and “victim” grating contrast, and inducing grating extent. Both effects cause “victim” gratings to be matched non-veridically. The respective magnitude of the effects was measured in common units which indexed the veridicality of contrast matches across a wide range (-0.9 to 0.9) of “victim” grating contrast. Grating induction had a low-pass, and contrast-contrast had a high-pass spatial frequency response, crossing over at ca. 1 c/d. Maximal grating induction strength exceeded that of contrast–contrast for 5 of 6 observers. Observers demonstrating strong grating induction tended to show weak contrast-contrast magnitudes, and vice versa. When inducing contrast was 0.75, the departures from veridical matching varied with “victim” grating contrast. For low frequencies (0.03125-0.125 c/d...
Illusory, or induced, brightness phenomena have for many years interested vision scientists becau... more Illusory, or induced, brightness phenomena have for many years interested vision scientists because they offer the potential to reveal fundamental truths concerning the mechanisms of brightness and contrast processing. The traditional idea that such phenomena reflect the operation of “lateral inhibition”, a term which predates and is cognate to the modern usage of “bandpass filtering”, has recently come under attack from a number of quarters. It has been shown for instance, in the classic demonstration of simultaneous contrast in which two identical grey patches appear markedly different in brightness depending on the luminance of their backgrounds, that the magnitude of the brightness difference between the grey patches depends on whether the viewer perceives the backgrounds to be of different reflectance and thus identically illuminated, or of identical reflectance and thus differently illuminated (Gilchrist, 1977, 1979; see also Knin & Kersten, 1991; Arend & Spehar, 1993a,b; Adel...
The White effect is an illusion in which gray test patches of identical luminance placed on the b... more The White effect is an illusion in which gray test patches of identical luminance placed on the black and white bars of a square-wave grating appear different in brightness/lightness. The effect has received much attention because the direction of the brightness change does not correlate with the amount of black or white border in contact with the gray test patch or its general vicinity. The test patch on the black bar appears lighter than the test patch on the white bar despite changes in test patch height or inducing grating spatial frequency. In addition, although the test patch shows a smooth change in brightness/lightness as its spatial position is varied relative to the inducing grating, spatial inhomogeneities in brightness/lightness within the test patch are also visible. A large number of “higher-level” explanations have been offered for the White effect; only the oriented-difference-of-Gaussians model can account for all of these properties.
Grating induction is a brightness/lightness illusion in which a sinewave luminance grating induce... more Grating induction is a brightness/lightness illusion in which a sinewave luminance grating induces the appearance of a counterphase sinusoidal grating in a homogeneous test field oriented orthogonally to the inducing grating. Induction is greatest at low spatial and temporal frequencies and declines with increasing frequency in both dimensions. Induction magnitude also declines with increasing test field height and scales as the product of inducing grating spatial frequency (c/d) and test field height. These properties of grating induction are difficult to explain using nonfiltering-based models but are readily accounted for by multiscale spatial filtering and lend support to such models of brightness/lightness induction.
What visual illusions tell us about underlying neural mechanisms and observer strategies for tack... more What visual illusions tell us about underlying neural mechanisms and observer strategies for tackling the inverse problem of achromatic perception. Front. Hum. Neurosci. 9:205. doi: 10.3389/fnhum.2015.00205 What visual illusions tell us about underlying neural mechanisms and observer strategies for tackling the inverse problem of achromatic perception
I am a research-oriented applications developer with more than ten years of professional programm... more I am a research-oriented applications developer with more than ten years of professional programming experience primarily in the area of cognitive and vision research. I have been working at the Center for Visual and Cognitive Neuroscience at North Dakota State University since 2005. I have expertise in mathematics, software development, and hardware and software, trouble-shooting. I have had experience with computer programming from the time when FORTRAN was popular as a language and personal computers were just being introduced. As an undergraduate, programming was a tool to augment my interests in biology and psychology. In graduate school, I had the opportunity to hone my skills, and formalize my computer science and neuroscience education. Since then, I have had experience implementing experiments involving virtual immersion, EEG recording, gaze tracking, image processing, statistical analyses, and various types of computer modelling.
IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2018
Embedded memory is a critical component in today's mobile video processing systems, increasingly ... more Embedded memory is a critical component in today's mobile video processing systems, increasingly dominating power consumption and shortening battery life of mobile devices. Traditional hardware-level power optimization techniques usually come with significant implementation overhead to solve the memory failure problem during low-voltage operations. This paper presents a novel mobile video memory to exploit the power saving opportunities provided by viewer experience under environmental visual interference. The viewing contexts, in particular the ambient luminance, significantly influence the quality of viewer experience, and in the context with higher luminance levels, mobile users have higher tolerance to the video degradation. Accordingly, the memory failures can be introduced adaptively to achieve power savings without influencing the viewer experience. To meet the silicon area constraint in mobile devices, a simple but efficient hardware implementation scheme is developed to minimize area overhead. The experimental results based on a 45-nm CMOS technology show that, as compared to the conventional memory design, the proposed technique can achieve up to 48% power savings with good perceivable quality and negligible implementation overhead.
Mobile devices are becoming ever more popular for streaming videos, which account for the majorit... more Mobile devices are becoming ever more popular for streaming videos, which account for the majority of all the data traffic on the Internet. Memory is a critical component in mobile video processing systems, increasingly dominating the power consumption. Today, memory designers are still focusing on hardware-level power optimization techniques, which usually come with significant implementation cost (e.g., silicon area overhead or performance penalty). In this paper, we propose a video content-aware memory technique for power-quality tradeoff from viewer's perspectives. Based on the influence of video macroblock characteristics on the viewer's experience, we develop two simple and effective models-decision tree and logistic regression to enable hardware adaptation. We have also implemented a novel viewer-aware bit-truncation technique which minimizes the impact on the viewer's experience, while introducing energy-quality adaptation to the video storage. INDEX TERMS Viewer's experience, video memory, video content, viewer-aware bit truncation, energy-quality adaptation.
2016 IEEE International Symposium on Circuits and Systems (ISCAS), 2016
The use of mobile devices is rapidly growing each year due to their portability and convenience. ... more The use of mobile devices is rapidly growing each year due to their portability and convenience. User experience and battery life are both crucial topics in the advancement of these devices. Sensing the light environment of a mobile device and adapting the output of the device accordingly can create positive results for both user experience and battery life. In this paper, we concentrate on different levels of ambient luminance and its influence on the viewing experience of the user. Since the human visual system is less sensitive to video noise at higher levels of ambient luminance, under those conditions we allow more noise to achieve power consumption reduction. Through subjective testing we show that in general our method works well for attaining a good quality of experience to the user. The hardware simulation for our design results in 32.6% power savings.
Comments and responses to "Theoretical approaches to lightness and perception" † A common theoret... more Comments and responses to "Theoretical approaches to lightness and perception" † A common theoretical framework for brightness and lightness perception In a review of the brightness/lightness literature, Kingdom (2011) noted that "Divided into different camps, each with its own preferred stimuli, methodology and theory, the study of LBT [lightness, brightness, transparency] is sometimes more reminiscent of the social sciences with its deep ideological divides than it is of the neurosciences" (page 652). Methodology and theory clearly separate our work from that of Gilchrist as described by Kingdom. The similarity of the underlying problems, however, has led us to investigate the roots of these differences and to propose a framework to bridge the divide and advance the field (Blakeslee & McCourt, 2015). The common question for brightness/lightness research is: how, and under what circum stances, is the visual system able to differentiate the physically invariant reflectance of a surface from its potentially changing illumination? The visual system lacks direct access to either quantity, being given only their product which determines the retinal luminance (intensity) distribution. The recovery of surface reflectance and illumination is thus an inverse problem with no algorithmic solution, and observers must therefore employ knowledge or prior assumptions (conscious or unconscious, learned or innate) about these variables to approximate a solution. We propose a framework for brightness/lightness research that resolves many of the confusions that have plagued communication between research groups (for details see Blakeslee & McCourt, 2015). The core ideas of this framework are that: (1) much of the confusion in the brightness/lightness literature stems from groups using different definitions of the central variables (brightness and lightness); (2) the term 'lightness', when defined simply as apparent reflectance, is underspecified with regard to illumination and is often used to refer to three independent and incomparable types of judgments. We discuss the above confusions and the false dichotomy between brightness and lightness research paradigms that they create, in relation to the preceding paper (Gilchrist, 2015). † Gilchrist, A. (2015). Theoretical approaches to lightness and perception. Perception, 44, this issue.
A variety of stimulus factors have been shown to influence the degree of leftward displacement of... more A variety of stimulus factors have been shown to influence the degree of leftward displacement of perceived line midpoint (i.e., pseudoneglect), which typifies the performance of normal subjects in line bisection tasks [M.E.
The brightness (perceived intensity) of a region of visual space depends on its luminance and on ... more The brightness (perceived intensity) of a region of visual space depends on its luminance and on the luminance of nearby regions. This phenomenon is called brightness induction and includes both brightness contrast and assimilation. Historically, and on a purely descriptive level, brightness contrast refers to a directional shift in target brightness away from the brightness of an adjacent region while assimilation refers to a brightness shift toward that of an adjacent region. In order to understand mechanisms, it is important to differentiate the descriptive terms contrast and assimilation from the optical and/or neural processes, often similarly named, which cause the effects. Experiment 1 isolated the effect on target patch (64 cd/m2) matching luminance (brightness) of six surround-ring widths (0.1°–24.5°) varied over 11 surround-ring luminances (32–96 cd/m2). Using the same observers, Experiment 2 examined the effect of the identical surround-ring parameters on target patch mat...
Two supra-threshold lateral interactions, the grating-induction effect,1 and the contrast–contras... more Two supra-threshold lateral interactions, the grating-induction effect,1 and the contrast–contrast effect,2,3 were compared regarding their dependence upon inducing grating spatial frequency, inducing and “victim” grating contrast, and inducing grating extent. Both effects cause “victim” gratings to be matched non-veridically. The respective magnitude of the effects was measured in common units which indexed the veridicality of contrast matches across a wide range (-0.9 to 0.9) of “victim” grating contrast. Grating induction had a low-pass, and contrast-contrast had a high-pass spatial frequency response, crossing over at ca. 1 c/d. Maximal grating induction strength exceeded that of contrast–contrast for 5 of 6 observers. Observers demonstrating strong grating induction tended to show weak contrast-contrast magnitudes, and vice versa. When inducing contrast was 0.75, the departures from veridical matching varied with “victim” grating contrast. For low frequencies (0.03125-0.125 c/d...
Illusory, or induced, brightness phenomena have for many years interested vision scientists becau... more Illusory, or induced, brightness phenomena have for many years interested vision scientists because they offer the potential to reveal fundamental truths concerning the mechanisms of brightness and contrast processing. The traditional idea that such phenomena reflect the operation of “lateral inhibition”, a term which predates and is cognate to the modern usage of “bandpass filtering”, has recently come under attack from a number of quarters. It has been shown for instance, in the classic demonstration of simultaneous contrast in which two identical grey patches appear markedly different in brightness depending on the luminance of their backgrounds, that the magnitude of the brightness difference between the grey patches depends on whether the viewer perceives the backgrounds to be of different reflectance and thus identically illuminated, or of identical reflectance and thus differently illuminated (Gilchrist, 1977, 1979; see also Knin & Kersten, 1991; Arend & Spehar, 1993a,b; Adel...
The White effect is an illusion in which gray test patches of identical luminance placed on the b... more The White effect is an illusion in which gray test patches of identical luminance placed on the black and white bars of a square-wave grating appear different in brightness/lightness. The effect has received much attention because the direction of the brightness change does not correlate with the amount of black or white border in contact with the gray test patch or its general vicinity. The test patch on the black bar appears lighter than the test patch on the white bar despite changes in test patch height or inducing grating spatial frequency. In addition, although the test patch shows a smooth change in brightness/lightness as its spatial position is varied relative to the inducing grating, spatial inhomogeneities in brightness/lightness within the test patch are also visible. A large number of “higher-level” explanations have been offered for the White effect; only the oriented-difference-of-Gaussians model can account for all of these properties.
Grating induction is a brightness/lightness illusion in which a sinewave luminance grating induce... more Grating induction is a brightness/lightness illusion in which a sinewave luminance grating induces the appearance of a counterphase sinusoidal grating in a homogeneous test field oriented orthogonally to the inducing grating. Induction is greatest at low spatial and temporal frequencies and declines with increasing frequency in both dimensions. Induction magnitude also declines with increasing test field height and scales as the product of inducing grating spatial frequency (c/d) and test field height. These properties of grating induction are difficult to explain using nonfiltering-based models but are readily accounted for by multiscale spatial filtering and lend support to such models of brightness/lightness induction.
What visual illusions tell us about underlying neural mechanisms and observer strategies for tack... more What visual illusions tell us about underlying neural mechanisms and observer strategies for tackling the inverse problem of achromatic perception. Front. Hum. Neurosci. 9:205. doi: 10.3389/fnhum.2015.00205 What visual illusions tell us about underlying neural mechanisms and observer strategies for tackling the inverse problem of achromatic perception
I am a research-oriented applications developer with more than ten years of professional programm... more I am a research-oriented applications developer with more than ten years of professional programming experience primarily in the area of cognitive and vision research. I have been working at the Center for Visual and Cognitive Neuroscience at North Dakota State University since 2005. I have expertise in mathematics, software development, and hardware and software, trouble-shooting. I have had experience with computer programming from the time when FORTRAN was popular as a language and personal computers were just being introduced. As an undergraduate, programming was a tool to augment my interests in biology and psychology. In graduate school, I had the opportunity to hone my skills, and formalize my computer science and neuroscience education. Since then, I have had experience implementing experiments involving virtual immersion, EEG recording, gaze tracking, image processing, statistical analyses, and various types of computer modelling.
IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2018
Embedded memory is a critical component in today's mobile video processing systems, increasingly ... more Embedded memory is a critical component in today's mobile video processing systems, increasingly dominating power consumption and shortening battery life of mobile devices. Traditional hardware-level power optimization techniques usually come with significant implementation overhead to solve the memory failure problem during low-voltage operations. This paper presents a novel mobile video memory to exploit the power saving opportunities provided by viewer experience under environmental visual interference. The viewing contexts, in particular the ambient luminance, significantly influence the quality of viewer experience, and in the context with higher luminance levels, mobile users have higher tolerance to the video degradation. Accordingly, the memory failures can be introduced adaptively to achieve power savings without influencing the viewer experience. To meet the silicon area constraint in mobile devices, a simple but efficient hardware implementation scheme is developed to minimize area overhead. The experimental results based on a 45-nm CMOS technology show that, as compared to the conventional memory design, the proposed technique can achieve up to 48% power savings with good perceivable quality and negligible implementation overhead.
Mobile devices are becoming ever more popular for streaming videos, which account for the majorit... more Mobile devices are becoming ever more popular for streaming videos, which account for the majority of all the data traffic on the Internet. Memory is a critical component in mobile video processing systems, increasingly dominating the power consumption. Today, memory designers are still focusing on hardware-level power optimization techniques, which usually come with significant implementation cost (e.g., silicon area overhead or performance penalty). In this paper, we propose a video content-aware memory technique for power-quality tradeoff from viewer's perspectives. Based on the influence of video macroblock characteristics on the viewer's experience, we develop two simple and effective models-decision tree and logistic regression to enable hardware adaptation. We have also implemented a novel viewer-aware bit-truncation technique which minimizes the impact on the viewer's experience, while introducing energy-quality adaptation to the video storage. INDEX TERMS Viewer's experience, video memory, video content, viewer-aware bit truncation, energy-quality adaptation.
2016 IEEE International Symposium on Circuits and Systems (ISCAS), 2016
The use of mobile devices is rapidly growing each year due to their portability and convenience. ... more The use of mobile devices is rapidly growing each year due to their portability and convenience. User experience and battery life are both crucial topics in the advancement of these devices. Sensing the light environment of a mobile device and adapting the output of the device accordingly can create positive results for both user experience and battery life. In this paper, we concentrate on different levels of ambient luminance and its influence on the viewing experience of the user. Since the human visual system is less sensitive to video noise at higher levels of ambient luminance, under those conditions we allow more noise to achieve power consumption reduction. Through subjective testing we show that in general our method works well for attaining a good quality of experience to the user. The hardware simulation for our design results in 32.6% power savings.
Comments and responses to "Theoretical approaches to lightness and perception" † A common theoret... more Comments and responses to "Theoretical approaches to lightness and perception" † A common theoretical framework for brightness and lightness perception In a review of the brightness/lightness literature, Kingdom (2011) noted that "Divided into different camps, each with its own preferred stimuli, methodology and theory, the study of LBT [lightness, brightness, transparency] is sometimes more reminiscent of the social sciences with its deep ideological divides than it is of the neurosciences" (page 652). Methodology and theory clearly separate our work from that of Gilchrist as described by Kingdom. The similarity of the underlying problems, however, has led us to investigate the roots of these differences and to propose a framework to bridge the divide and advance the field (Blakeslee & McCourt, 2015). The common question for brightness/lightness research is: how, and under what circum stances, is the visual system able to differentiate the physically invariant reflectance of a surface from its potentially changing illumination? The visual system lacks direct access to either quantity, being given only their product which determines the retinal luminance (intensity) distribution. The recovery of surface reflectance and illumination is thus an inverse problem with no algorithmic solution, and observers must therefore employ knowledge or prior assumptions (conscious or unconscious, learned or innate) about these variables to approximate a solution. We propose a framework for brightness/lightness research that resolves many of the confusions that have plagued communication between research groups (for details see Blakeslee & McCourt, 2015). The core ideas of this framework are that: (1) much of the confusion in the brightness/lightness literature stems from groups using different definitions of the central variables (brightness and lightness); (2) the term 'lightness', when defined simply as apparent reflectance, is underspecified with regard to illumination and is often used to refer to three independent and incomparable types of judgments. We discuss the above confusions and the false dichotomy between brightness and lightness research paradigms that they create, in relation to the preceding paper (Gilchrist, 2015). † Gilchrist, A. (2015). Theoretical approaches to lightness and perception. Perception, 44, this issue.
A variety of stimulus factors have been shown to influence the degree of leftward displacement of... more A variety of stimulus factors have been shown to influence the degree of leftward displacement of perceived line midpoint (i.e., pseudoneglect), which typifies the performance of normal subjects in line bisection tasks [M.E.
Uploads
Papers by Mark McCourt