Inferring word relevance from eye-movements of readers
Peter Brusilovsky2011
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
Reading is one of the most important skills in today's society. The ubiquity of this activity has naturally affected many information systems; the only goal of some is the presentation of textual information. One concrete task often performed on a computer and involving reading is finding relevant parts of text. In the current study, we investigated if word-level relevance, defined as a binary measure of an individual word being congruent with the reader's current informational needs, could be inferred given only the text and eye movements of readers. We found that the number of fixations, first-pass fixations, and the total viewing time can be used to predict the relevance of sentence-terminal words. In light of what is known about eye movements of readers, knowing which sentence-terminal words are relevant can help in an unobtrusive identification of relevant sentences.
Related papers
Can Relevance Be Inferred From Eye Movements In Information Retrieval
Proceedings of WSOM, 2003
Measuring the Impact of (Psycho-)Linguistic and Readability Features and Their Spill Over Effects on the Prediction of Eye Movement Patterns
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
There is a growing interest in the combined use of NLP and machine learning methods to predict gaze patterns during naturalistic reading. While promising results have been obtained through the use of transformer-based language models, little work has been undertaken to relate the performance of such models to general text characteristics. In this paper we report on experiments with two eye-tracking corpora of naturalistic reading and two language models (BERT and GPT-2). In all experiments, we test effects of a broad spectrum of features for predicting human reading behavior that fall into five categories (syntactic complexity, lexical richness, register-based multiword combinations, readability and psycholinguistic word properties). Our experiments show that both the features included and the architecture of the transformer-based language models play a role in predicting multiple eye-tracking measures during naturalistic reading. We also report the results of experiments aimed at determining the relative importance of features from different groups using SP-LIME.
Eye movements in reading words and sentences
… : A window on mind and brain, 2007
Relevance feedback from eye movements for proactive information retrieval
2004
We study whether it is possible to infer from eye movements measured during reading what is relevant for the user in an information retrieval task. Inference is made using hidden Markov and discriminative hidden Markov models. The result of this feasibility study is that prediction of relevance is possible to a certain extent, and models benefit from taking into account the time series nature of the data.
The reading process of dynamic text – A linguistic approach to an eye movement study
SKY Journal of Linguistics, 2016
Using eye-movement analysis, the article examines the reading process of speech-to-text interpretation involving dynamic text emerging letter by letter on the screen. The article focuses on regressions of gaze as well as on their relationship to linguistic factors in order to reveal how the reader's gaze behaviour reflects the reading process of dynamic text. The data come from an experiment where participants read a dynamic text on a computer screen. The results showed that the first and second landing points of regressions were generally (90.8%) content words, even though the proportion of content words in the whole data set was only 57.1%. The test subjects looked for nouns, verbs and adjectives in order to construct the meaning of what they had just read. Nouns were the most likely landing points of regressions. The landing points of regressions reflected the reading process through which the meaning of the text was constructed. In this kind of dynamic text, a typical cause of regressions seems to be incoherence resulting from omissions.
Skim reading by satisficing: Evidence from eye tracking
CHI '11
Readers on the Web often skim through text to cope with the volume of available information. In a previous study [11] readers’ eye movements were tracked as they skimmed through expository text under time pressure. This article presents novel analyses of these eye-movement data. Results indicated that readers were able to explicitly direct attention to the most important information in the text and that this improved performance on a subsequent test of memory for the meaning of text. We suggest readers achieve this by satisficing – reading through text until the rate of information gain drops below threshold and then skipping to the next section of text. Further analyses of gaze patterns for paragraphs and pages supported this explanation. Combining satisficing with some form of scanning or sampling behaviour could explain patterns of reading found on the Web. A greater understanding of the way that text is read on the Web would assist many producers of online content.
Feedback beyond accuracy: Using eye‐tracking to detect comprehensibility and interest during reading
Journal of the Association for Information Science and Technology, 2023
Knowing what information a user wants is a paramount challenge to information science and technology. Implicit feedback is key to solving this challenge, as it allows information systems to learn about a user's needs and preferences. The available feedback, however, tends to be limited and its interpretation shows to be difficult. To tackle this challenge, we present a user study that explores whether tracking the eyes can unpack part of the complexity inherent to relevance and relevance decisions. The eye behavior of 30 participants reading 18 news articles was compared with their subjectively appraised comprehensibility and interest at a discourse level. Using linear regression models, the eye‐tracking signal explained 49.93% (comprehensibility) and 30.41% (interest) of variance (p < .001). We conclude that eye behavior provides implicit feedback beyond accuracy that enables new forms of adaptation and interaction support for personalized information systems.
Eye Movements and Measures of Reading Time
Eye Guidance in Reading and Scene Perception, 1998
In this chapter, we consider the use of reading time measures that sum fixation durations in order to gain an understanding of the difficulty experienced when reading texts. We draw a distinction between two approaches to summing the duration of fixations. One approach is to sum the duration of fixations that are spatially contiguous in the text, meaning that the fixations neighbour each other in a specified region of space. The other approach is to sum fixations that are temporally contiguous, meaning that they occur in a sequence over a specified period of time. It is argued that both types of reading time measure are needed if the experimenter is to understand the time course of the influence of a linguistic variable on readers' processing of text. We first discuss a number of hypothetical eye movement records in order to illustrate the differential sensitivities of qualitatively different eye movement measures. We then report an eye movement experiment investigating how people process reduced relative clause sentences with and without the focus operator only in order to examine the utility of different reading time measures. The results showed that measures summing temporally contiguous fixations can make an important contribution to the experimenters' understanding of the precise pattern of eye movements which occur when a problem is encountered in the text.
Attention-Based Information Retrieval Using Eye Tracker Data
Proceedings of the 3rd International Conference on Knowledge Capture (K-CAP 2005), 2005
We describe eFISK, an automated keyword extraction system which unobtrusively measures the user's attention in order to isolate and identify those areas of a written document the reader finds of greatest interest. Attention is measured by use of eye-tracking hardware consisting of a desk-mounted infrared camera which records various data about the user's eye. The keywords thus identified are subsequently used in the back end of an information retrieval system to help the user find other documents which contain information of interest to him. Unlike traditional IR techniques which compare documents simply on the basis of common terms withal, our system also accounts for the weights users implicitly attach to certain words or sections of the source document. We describe a task-based user study which compares the utility of standard relevance feedback techniques to the keywords and keyphrases discovered by our system in finding other relevant documents from a corpus.
Ocular-based automatic summarization of documents : is re-reading informative about the importance of a sentence ?
Automatic document summarization (ADS) has been introduced as a viable solution for reducing the time and the effort needed to read the ever-increasing textual content that is disseminated. However, a successful universal ADS algorithm has not yet been developed. Also, despite progress in the field, many ADS techniques do not take into account the needs of different readers, providing a summary without internal consistency and the consequent need to re-read the original document. The present study was aimed at investigating the usefulness of using eye tracking for increasing the quality of ADS. The general idea was of that of finding ocular behavioural indicators that could be easily implemented in ADS algorithms. For instance, the time spent in re-reading a sentence might reflect the relative importance of that sentence, thus providing a hint for the selection of text contributing to the summary. We have tested this hypothesis by comparing metrics based on the analysis of eye movem...
Related papers
Automated and interactive analysis of eye-fixation data in reading
A package of five FORTRAN programs that provides for fast user-controlled analyses of reading eye fixations is described. The package requires the data to be in a fixation format and to be rescaled to screen dimensions. OLDEYE identifies six types of fixations and calculates descriptive statistics on each of them, on their associated saccades, and on their average pupil diameter. CONVRT represents the text as a string of words that can be coded according to experimentally relevant variables. PLTFIX prints fixation durations by letter position and sequence of occurrence. MODDAT is an interactive program for marking parts of the text in which the data quality is below acceptable standards. It also allows the correction of systematic errors due to calibration or drift. MATCH combines the outputs from OLDEYE, CONVRT, and MODDAT and calculates 11 dependent measures for every word. The output of MATCH is suitable for input to conventional multivariate statistical program
Towards a flexible, reusable model for predicting eye movements during visual search of text
2006
Abstract Visual search is an integral component in many human activities. The eye movements produced during such activities can provide valuable information about people's cognitive processes. This research investigates, with detailed eye movement data analysis and computational cognitive modeling, the perceptual, strategic, and oculomotor processes people use to visually search. A cognitive model is evolved in a principled manner based on eye movement data, past modeling efforts, and recent psychological literature.
Eye movement measures to study global text processing
The mind's eye: Cognitive and …, 2003
In this chapter, we demonstrate the usefulness of the eye tracking method in studying global text processing. By "global text processing," we refer to processes responsible for the integration of information from sentences that are not adjacent in the text. Potential eye movement measures indexing global text processing are discussed using as examples the processing of topic-introducing sentences and the processing of inconsistencies. In addition to the existing measures of regional gaze duration and lookback fixation time, we advocate new measures that may be applied to the study of global text processing. These include a new extended first-pass fixation time measure that allows lookbacks to previous text regions without necessarily terminating the first-pass reading, and first-pass rereading time that sums up all the reinspective fixations made during first-pass reading. We also demonstrate the potential usefulness of analyzing the origin and destination of eye movement sequences, such as lookback sequences.
Toward a model of eye movement control in reading
Psychological Review, 1998
The authors present several versions of a general model, titled the E-Z Reader model, of eye movement control in reading. The major goal of the modeling is to relate cognitive processing (specifically aspects of lexical access) to eye movements in reading. The earliest and simplest versions of the model (E-Z Readers 1 and 2) merely attempt to explain the total time spent on a word before moving forward (the gaze duration) and the probability of fixating a word; later versions (E-Z Readers 3-5) also attempt to explain the durations of individual fixations on individual words and the number of fixations on individual words. The final version (E-Z Reader 5) appears to be psychologically plausible and gives a good account of many phenomena in reading. It is also a good tool for analyzing eye movement data in reading. Limitations of the model and directions for future research are also discussed.
Learning where to look: modeling eye movements in reading
We propose a novel machine learning task that consists in learning to predict which words in a text are fixated by a reader. In a first pilot experiment, we show that it is possible to outperform a majority baseline using a transitionbased model with a logistic regression classifier and a very limited set of features. We also show that the model is capable of capturing frequency effects on eye movements observed in human readers.
Eye Gaze Relevance Feedback Indicators for Information Retrieval
International Journal of Intelligent Systems and Applications, 2022
There is a growing interest in the research on interactive information retrieval, particularly in the study of eye gaze-enhanced interaction. Feedback generated from user gaze features is important for developing an interactive information retrieval system. Generating these gaze features have become less difficult with the advancement of the eye tracker system over the years. In this work, eye movement as a source of relevant feedback was examined. A controlled user experiment was carried out and a set of documents were given to users to read before an eye tracker and rate the documents according to how relevant they are to a given task. Gaze features such as fixation duration, fixation count and heat maps were captured. The result showed a medium linear relationship between fixation count and user explicit ratings. Further analysis was carried out and three classifiers were compared in terms of predicting document relevance based on gaze features. It was found that the J48 decision...
The influence of focus on eye movements during reading
In this chapter, we review research investigating the influence of linguistic focus on eye movements during reading. Focus is the assignment of prominence by phonological or syntactic means, which either marks new information or indicates that a contrast should be made between the focused information and some alternatives to it. We show that focus has a wide ranging influence on reading, from the processing of individual words, to the processing of sentence structure, the computation of reference, and the nature of the mental representation constructed by a reader. We conclude that eye movement research has been successful in revealing readers’ sensitivity to focus during language comprehension, and its influence on higher-order language processing during the reading and understanding of text.
Contributions of reader- and text-level characteristics to eye-movement patterns during passage reading
Journal of Experimental Psychology: Learning, Memory and Cognition, 2018
The present research presents a novel method for investigating how characteristics of texts (words, sentences and passages) and individuals (verbal and general cognitive skills) jointly influence eyemovement patterns over the time-course of reading, as well as comprehension accuracy. Fifty-one proficient readers read passages of varying complexity from the Gray Oral Reading Test, while their eye-movements were recorded. Participants also completed a large battery of tests assessing various components of reading comprehension ability (vocabulary size, decoding, phonological awareness, and experience with print), as well as general cognitive and executive skills. We used the Random Forests non-parametric regression technique to simultaneously estimate relative importance of all predictors. This method enabled us to trace the temporal engagement of individual predictors and entire predictor groups on eye-movements during reading, while avoiding the problems of model overfitting and collinearity, typical of parametric regression methods. Our findings both confirmed well-established results of prior research and pointed to a space of hypotheses that is as yet unexplored. Keywords random forests; eye movements; reading; individual differences Eye-movements during passage reading are susceptible to at least three sources of variability, stemming from i) the cognitive and linguistic ability of the reader him/herself; ii) linguistic properties of the text itself; and iii) the dynamic requirements of the reading task itself. While the first two have been well studied in the literature, they are typically not examined jointly (but see Rayner, 1998, 2009 and the literature review below). The third, which requires the coordinated uptake of perceptual information (i.e., identification of lines and circles that constitute symbols) as well as the timely integration of various levels of information in the process of creating a coherent meaning representation, has only recently received direct attention (e.g., Goswami, 2011) but this work has not focused on eyemovements as a gateway for information uptake. These three sources-labeled here as Reader, Text, and Time-are known to interact (see the literature review below), and thus
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.