Handbook of Latent Semantic Analysis 1st Edition

Thomas K. Landauer digital download

Now available at ebookultra.com

( 4.4/5.0 ★ | 468 downloads )

https://ebookultra.com/download/handbook-of-latent-semantic-
analysis-1st-edition-thomas-k-landauer/
Handbook of Latent Semantic Analysis 1st Edition Thomas K.
Landauer

EBOOK

Available Formats

■ PDF eBook Study Guide Ebook

EXCLUSIVE 2025 ACADEMIC EDITION – LIMITED RELEASE

Available Instantly Access Library

Here are some recommended products for you. Click the link to
download, or explore more at ebookultra.com

Applied Latent Class Analysis 1st Edition Jacques A.

Hagenaars

https://ebookultra.com/download/applied-latent-class-analysis-1st-
edition-jacques-a-hagenaars/

Semantic Multimedia Analysis and Processing 1st Edition

Evaggelos Spyrou

https://ebookultra.com/download/semantic-multimedia-analysis-and-
processing-1st-edition-evaggelos-spyrou/

Handbook of latent variable and related models 1st ed

Edition Sik-Yum Lee

https://ebookultra.com/download/handbook-of-latent-variable-and-
related-models-1st-ed-edition-sik-yum-lee/

The Handbook of Critical Intercultural Communication

Handbooks in Communication and Media 1st Edition Thomas K.
Nakayama
https://ebookultra.com/download/the-handbook-of-critical-
intercultural-communication-handbooks-in-communication-and-media-1st-
edition-thomas-k-nakayama/
Explorer s Guide to the Semantic Web 1st Edition Thomas B.
Passin

https://ebookultra.com/download/explorer-s-guide-to-the-semantic-
web-1st-edition-thomas-b-passin/

SchenkerGUIDE a brief handbook and website for Schenkerian

analysis 1st Edition Thomas Pankhurst

https://ebookultra.com/download/schenkerguide-a-brief-handbook-and-
website-for-schenkerian-analysis-1st-edition-thomas-pankhurst/

Semantic Network Analysis in Social Sciences 1st Edition

Elad Segev (Editor)

https://ebookultra.com/download/semantic-network-analysis-in-social-
sciences-1st-edition-elad-segev-editor/

Revolution and Other Writings Gustav Landauer

https://ebookultra.com/download/revolution-and-other-writings-gustav-
landauer/

Handbook of Seismic Risk Analysis and Management of Civil

Infrastructure Systems 1st Edition S. Tesfamariam And K.
Goda (Eds.)
https://ebookultra.com/download/handbook-of-seismic-risk-analysis-and-
management-of-civil-infrastructure-systems-1st-edition-s-tesfamariam-
and-k-goda-eds/
Handbook of Latent
Semantic Analysis
Handbook of Latent
Semantic Analysis

Edited by
Thomas K Landauer
Danielle S. McNamara
Simon Dennis
Walter Kintsch
First published by
Lawrence Erlbaum Associates, Inc.
10 Industrial Avenue
Mahwah, New Jersey 07430

This edition published 2011 by Routledge

Routledge Routledge
Taylor & Francis Group Taylor & Francis Group
711 Third Avenue 2 Park Square, Milton Park
New York, NY 10017 Abingdon, Oxon OX14 4RN

Copyright © 2007 by Lawrence Erlbaum Associates, Inc.

All rights reserved. No part of this book may be reproduced in any
form, by photostat, microform, retrieval system, or any other
means, without prior written permission of the publisher.

Cover design by Tomai Maridou

Cover art “Round About,” 1998, 48” x 48”, Oil Paint and Roofing
Tar on Denim, by Randy McNamara

Library of Congress Cataloging-in-Publication Data

Handbook of latent semantic analysis / edited by Thomas K Landauer …

[et al.].
p. cm. — (University of Colorado Institute of Cognitive Science series)
Includes bibliographical references and index.
ISBN 978-0-8058-5418-3 — 0-8058-5418-5 (cloth : alk. paper)
ISBN 978-1-4106-1534-3 — 1-4106-1534-0 (E Book)
1. Semantics—Data processing. 2. Semantics—Mathematical models. 3.
Semantics—Psychological aspects. I. Landauer, Thomas K
P325.5.D38H36 2006
410’.151—dc22 2006014971
CIP
 Contents

Preface ix

PART I: INTRODUCTION TO LSA: THEORY

AND METHODS

1 LSA as a Theory of Meaning 3

Thomas K Landauer

2 Mathematical Foundations Behind Latent Semantic 35

Analysis
Dian I. Martin and Michael W. Berry

3 How to Use the LSA Web Site 57

Simon Dennis

4 Creating Your Own LSA Spaces 71

Jose Quesada

v
vi CONTENTS

PART II: LSA IN COGNITIVE THEORY

5 Meaning in Context 89
Walter Kintsch

6 Symbolic or Embodied Representations: A Case for 107

Symbol Interdependency
Max M. Louwerse

7 Semantic Structure and Episodic Memory 121

Marc W. Howard, Bing Jing, Kelly M. Addis,
and Michael J. Kahana

8 A Semantic Space for Modeling Children’s 143

Semantic Memory
Guy Denhière, Benoît Lemaire, Cédrick Bellissens,
and Sandra Jhean-Larose

9 Discourse Coherence and LSA 167

Peter W. Foltz

10 Spaces for Problem Solving 185

Jose Quesada

PART III: LSA IN EDUCATIONAL APPLICATIONS

11 Assessing and Improving Comprehension With Latent 207

Semantic Analysis
Keith Millis, Joseph Magliano, Katja Wiemer-Hastings,
Stacey Todaro, and Danielle S. McNamara

12 Evaluating Self-Explanations in iSTART: Comparing 227

Word-Based and LSA Algorithms
Danielle S. McNamara, Chutima Boonthum, Irwin
Levinstein, and Keith Millis

13 Using LSA in AutoTutor: Learning Through 243

Mixed-Initiative Dialogue in Natural Language
Art Graesser, Phani Penumatsa, Matthew Ventura,
Zhiqiang Cai, and Xiangen Hu
CONTENTS vii

14 Summary Street®: Computer-Guided Summary Writing 263

Eileen Kintsch, Donna Caccamise, Marita Franzke, Nina
Johnson, and Scott Dooley

15 Automated Tools for Collaborative Learning 279

Environments
Lynn A. Streeter, Karen E. Lochbaum, Noelle LaVoie,
and Joseph E. Psotka

PART IV: INFORMATION RETRIEVAL AND HCI

APPLICATIONS OF LSA

16 LSA and Information Retrieval: Getting Back to Basics 293

Susan T. Dumais

17 Helping People Find and Learn From Documents: 323

Exploiting Synergies Between Human and
®
Computer Retrieval With SuperManual
Peter W. Foltz and Thomas K Landauer

18 Automating Usability Evaluation: Cognitive Walkthrough 345

for the Web Puts LSA to Work on Real-World HCI
Design Problems
Marilyn Hughes Blackmon, Dipti R. Mandalia, Peter G.
Polson, and Muneo Kitajima

PART V: EXTENSIONS TO LSA

19 Optimizing LSA Measures of Cohesion 379

Danielle S. McNamara, Zhiqiang Cai, and Max M. Louwerse

20 Strengths, Limitations, and Extensions of LSA 401

Xiangen Hu, Zhiqiang Cai, Peter Wiemer-Hastings,
Art C. Graesser, and Danielle S. McNamara

21 Probabilistic Topic Models 427

Mark Steyvers and Tom Griffiths

22 Introducing Word Order Within the LSA Framework 449

Simon Dennis
viii CONTENTS

PART VI: CONCLUSION

23 LSA and Meaning: In Theory and Application 467

Walter Kintsch, Danielle S. McNamara, Simon Dennis,
and Thomas K Landauer

Author Index 481

Subject Index 491

 Preface

WHAT IS THIS BOOK FOR?

This volume, Handbook of Latent Semantic Analysis, celebrates the theoretical
and practical contributions of latent semantic analysis over the past 15 years.
These chapters, written by the leading investigators using LSA, provide the
research community with a seminal volume on this revolutionary technique.
The first editor, Tom Landauer, was the project leader of the team that devel-
oped LSA in the late 1980s. Since the time that LSA was first proposed, there
has been an explosion of research and applications involving LSA. LSA has
become both a widely debated theoretical model and a widely used cogni-
tive tool. A search for latent semantic analysis in Science Citation Index reveals
hundreds of published journal articles, most of which appeared in the last 3
years. In addition, the LSA Web site that provides access to the computa-
tional tools has received 7.8 million hits since April 2001.
Nonetheless, although there has been substantial interest in LSA, there
remain many misconceptions about the technical and theoretical underpin-
nings of the technique and how it can be appropriately used. Here we pro-
vide a definitive volume to which one can turn to gain a more thorough
understanding of LSA. The first section, Introduction to LSA: Theory and
Methods, provides the theoretical and mathematical foundations of LSA,
as well as basic information on how to use LSA. The second section, LSA in

ix
x PREFACE

Cognitive Theory, provides examples of how LSA can be used to test theo-
ries of cognition and to better understand cognitive phenomena. The third
and fourth sections, LSA in Educational Applications and Information Re-
trieval and HCI Applications of LSA, provide examples of how LSA can be
used in the real world. The fifth section describes ways in which LSA has
been extended to meet a wider array of problems and questions. Our goal is
to provide clear directives to researchers concerning the situations when
LSA will be the optimal technique to be used, and when it may not be.
Thus, this volume provides readers with valuable information about the
rationale and theory underlying LSA, basic information about how to use
LSA, implications of LSA for cognitive theories, and examples of how LSA
has been applied to practical problems. We hope this volume provides nec-
essary information and guidance about LSA and the examples of how LSA
has been used in practical applications will give others insight into how
they too can make use of what it offers. We further hope that discussion of
its theoretical assumptions, as well as current alternatives to LSA, will lead
to further developments of methods for extracting meaning from text. We
believe that LSA provides a road to meaning, and this destination will be
reached by communicating LSA to the research community.

WHAT IS LATENT SEMANTIC ANALYSIS?

Latent semantic analysis (LSA) is a theory and method for extracting and
representing the meaning of words. Meaning is estimated using statistical
computations applied to a large corpus of text (Landauer & Dumais, 1997).
The corpus embodies a set of mutual constraints that largely determine the
semantic similarity of words and sets of words. These constraints can be
solved using linear algebra methods, in particular, singular value decom-
position. LSA has been shown to reflect human knowledge in a variety of
ways. For example, LSA measures correlate highly with humans’ scores on
standard vocabulary and subject matter tests; it mimics human word sort-
ing and category judgments; it simulates word–word and passage–word
lexical priming data; and it accurately estimates passage coherence. In ad-
dition, LSA has found application in a number of areas, including selecting
educational materials for individual students, guiding online discussion
groups, providing feedback to pilots on landing technique, diagnosing
mental disorders from prose, matching jobs with candidates, and facilitat-
ing automated tutors.

LSA Is a Technique for Analyzing Text

Before the advent of LSA, extracting meaning from text involved significant
human intervention—legions of coders spending hours rewriting text in for-
mal notation. LSA analyses demonstrated that a great deal of what is con-
PREFACE xi

veyed by a text can be extracted automatically using the tools of linear

algebra. LSA provides not only that information explicit in the text, but also
the underlying or latent meaning of the text —an advance that has had pro-
found implications across a broad array of application areas.

LSA Is a Theory of Meaning

Whereas LSA was developed as a practical technique, it also makes novel
and hotly contested conjectures about the nature of meaning. The theory
postulates that the meaning of a text is largely conveyed by the words from
which it is composed. The order in which the words appear is important in
some specific cases, but it is not as critical in determining meaning as the
words themselves. Furthermore, the theory underlying LSA arguments
concerns the developmental acquisition of word meanings. Some theories
of vocabulary acquisition posit an incremental process that proceeds by
composing the elements of sense experience. In contrast, according to LSA,
meaning is acquired by solving an enormous set of simultaneous equations
that capture the contextual usage of words.

LSA Is a New Approach to Cognitive Science

Finally, LSA exemplifies a new approach to cognitive science. Rather than
focusing on dissecting the minutiae of laboratory experiments, LSA at-
tempts to account for the data produced by real participants in real tasks.
One task of cognitive science, it argues, should be to account for what most
people do most of the time. The use of large text corpora to test cognitive
theories is radically altering our assumptions and our approaches to an-
swering questions in cognitive science.

HOW DID THIS VOLUME COME ABOUT AND WHO

DO WE HAVE TO THANK?

This volume was preceded in May 2004 by a workshop at the University of

Colorado. We met there to discuss our research and to find common ground
among researchers using LSA. In that respect, the workshop was im-
mensely useful and illuminating. It was also fun. The workshop was
funded by the Institute of Cognitive Science at University of Colorado and
the Institute for Intelligent Systems at the University of Memphis; we are
grateful for the institutes’ support of such research endeavors. We are also
grateful to many individuals who helped to organize that workshop, in-
cluding the staff of the Institute of Cognitive Science and the many student
volunteers who helped make the conference both pleasant and successful.
We further thank the chapter reviewers for their dedication to the field. Al-
though the chapters were reviewed primarily by the contributors to this
xii PREFACE

volume, we also thank Mina Johnson-Glenberg and Daniel Navarro for

helping with the review process. Finally, we are most grateful to those who
contributed their chapters to this volume. Without the work that they have
conducted to use, explore, and understand LSA and in turn to better under-
stand human cognition, this volume most certainly would not have been
possible. We thank them for the research they are conducting and for their
contributions to this volume.
We also thank Randy McNamara for the artwork that graces the cover of
this book. Randy McNamara has an MFA from Indiana University, has won
numerous awards, and has shown extensively on the east and west coast,
most recently at Gallery 825 in Los Angeles. Randy McNamara presently
resides in Los Angeles with his wife and two young sons.

—Danielle McNamara
—Walter Kintsch
—Simon Dennis
—Tom Landauer

REFERENCES

Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato's problem: The Latent Se-
mantic Analysis theory of acquisition, induction and representation of knowl-
edge [Electronic version]. Psychological Review, 104(2), 211–240.
 I
Introduction to LSA
Theory and Methods
 1
LSA as a Theory of Meaning

Thomas K Landauer
Pearson Knowledge Technologies and University of Colorado

The fundamental scientific puzzle addressed by the latent semantic analy-

sis (LSA) theory is that there are hundreds of distinctly different human
languages, every one with tens of thousands of words. The ability to under-
stand the meanings of utterances composed of these words must be ac-
quired by virtually every human who grows up surrounded by language.
There must, therefore, be some humanly shared method—some computa-
tional system—by which any human mind can learn to do this for any lan-
guage by extensive immersion, and without being explicitly taught
definitions or rules for any significant number of words.
Most past and still popular discussions of the problem focus on debates
concerning how much of this capability is innate and how much learned
(Chomsky, 1991b) or what abstract architectures of cognition might sup-
port it—such as whether it rests on association (Skinner, 1957) or requires a
theory of mind (Bloom, 2000).
The issue with which LSA is concerned is different. LSA theory ad-
dresses the problem of exactly how word and passage meaning can be con-
s t ru c t e d f ro m e x p e r i e n c e w i t h l a n g u a g e , th a t i s , b y w h a t
mechanisms—instinctive, learned, or both—this can be accomplished.
Carefully describing and analyzing the phenomenon has been the center
of attention for experimental psychology, linguistics, and philosophy.
Other areas of interest include pinpointing what parts of the brain are most
3
4 LANDAUER

heavily involved in which functions and how they interact, or positing

functional modules and system models. But, although necessary or useful,
these approaches do not solve the problem of how it is possible to make the
brain, or any other system, acquire the needed abilities at their natural scale
and rate.
This leads us to ask the question: Suppose we have available a corpus of
data approximating the mass of intrinsic and extrinsic language-relevant
experience that a human encounters, a computer with power that could
match that of the human brain, and a sufficiently clever learning algorithm
and data storage method. Could it learn the meanings of all the words in
any language it was given?
The keystone discovery for LSA was that using just a single simple con-
straint on the structure of verbal meaning, and a rough approximation to
the same experience as humans, LSA can perform many meaning-based
cognitive tasks as well as humans.
That this provides a proof that LSA creates meaning is a proposition that
manifestly requires defense. Therefore, instead of starting with explication of
the workings of the model itself, the chapter first presents arguments in favor
of that proposition. The arguments rest on descriptions of what LSAachieves
and how its main counterarguments can be discounted.

THE TRADITIONAL ANTILEARNING ARGUMENT

Many well-known thinkers—Plato, Bickerton (1995), Chomsky (1991b),

Fodor (1987), Gleitman (1990), Gold (1967), Jackendoff (1992), Osherson,
Stob, and Weinstein (1984), Pinker (1994), to name a few—have considered
this prima facie impossible, usually on the grounds that humans learn lan-
guage too easily, that they are exposed to too little evidence, correction, or in-
struction to make all the conceptual distinctions and generalizations that
natural languages demand. This argument has been applied mainly to the
learning of grammar, but has been asserted with almost equal conviction to
apply to the learning of word meanings as well, most famously by Plato,
Chomsky, and Pinker. Given this postulate, it follows that the mind (brain, or
any equivalent computational system) must be equipped with other sources
of conceptual and linguistic knowledge. This is not an entirely unreasonable
hypothesis. After all, the vast majority of living things come equipped with
or can develop complex and important behavioral capabilities in isolation
from other living things. Given this widely accepted assumption, it would
obviously be impossible for a computer using input only from a sample of
natural language in the form of unmodified text to come even close to doing
things with verbal meaning that humans do.
1. LSA AS A THEORY OF MEANING 5

THE LSA BREAKTHROUGH

It was thus a major surprise to discover that a conceptually simple algo-

rithm applied to bodies of ordinary text could learn to match literate hu-
mans on tasks that if done by people would be assumed to imply
understanding of the meaning of words and passages. The model that first
accomplished this feat was LSA.
LSA is a computational model that does many humanlike things with
language. The following are but a few: After autonomous learning from a
large body of representative text, it scores well into the high school student
range on a standardized multiple-choice vocabulary test; used alone to rate
the adequacy of content of expository essays (other variables are added in
full- scale grading systems; Landauer, Laham, & Foltz, 2003a, 2003b), esti-
mated in more than one way, it shares 85%–90% as much information with
expert human readers as two human readers share with each other
(Landauer, 2002a); it has measured the effect on comprehension of para-
graph-to-paragraph coherence better than human coding (Foltz, Kintsch, &
Landauer, 1998); it has successfully modeled several laboratory findings in
cognitive psychology (Howard, Addis, Jing, & Kahana, chap. 7 in this vol-
ume; Landauer, 2002a; Landauer & Dumais, 1997; Lund, Burgess, &
Atchley, 1995); it detects improvements in student knowledge from before
to after reading as well as human judges (Rehder et al., 1998; Wolfe et al.,
1998); it can diagnose schizophrenia from what patients say as well as expe-
rienced psychiatrists (Elvevåg, Foltz, Weinberger, & Goldberg, 2005); it im-
proves information retrieval by up to 30% by being able to match queries to
documents of the same meaning when there are few or no words in com-
mon and reject those with many when irrelevant (Dumais, 1991), and can
do the same for queries in one language matching documents in another
where no words are alike (Dumais, Landauer, & Littman, 1996); it does its
basic functions of correctly simulating human judgments of meaning simi-
larity between paragraphs without modification by the same algorithm in
every language to which it has been applied, examples of which include
Arabic, Hindi, and Chinese in their native orthographic or ideographic
form; and when sets of all LSA similarities among words for perceptual en-
tities such as kinds of objects (e.g., flowers, trees, birds, chairs, or colors) are
subjected to multidimensional scaling, the resulting structures match those
based on human similarity judgments quite well in many cases, moder-
ately well in others (Laham, 1997, 2000), just as we would expect (and later
explain) because text lacks eyes, ears, and fingers.
I view these and its several other successful simulations (see Landauer,
2002a; Landauer, Foltz, & Laham, 1998) as evidence that LSA and models
like it (Griffiths & Steyvers, 2003; Steyvers & Griffiths, chap. 21 in this vol-
6 LANDAUER

ume) are candidate mechanisms to explain much of how verbal meaning

might be learned and used by the human mind.

ABOUT LSA’S KIND OF THEORY

LSA offers a very different kind of account of verbal meaning from any that
went before, including centuries of theories from philosophy, linguistics,
and psychology. Its only real predecessor is an explanation inherent in
connectionist models but unrealized yet at scale (O’Reilly & Munakata,
2000). Previous accounts had all been in the form of rules, descriptions, or
variables (parts of speech, grammars, etc.) that could only be applied by hu-
man intercession, products of the very process that needs explanation. By
contrast, at least in programmatic goal, the LSA account demands that the
only data allowed the theory and its computational instantiations be those
to which natural human language users have access. The theory must oper-
ate on the data by means that can be expressed with mathematical rigor, not
through the intervention of human judgments. This disallows any linguis-
tic rule or structure unless it can be proved that all human minds do equiva-
lent things without explicit instruction from other speakers, the long
unattained goal of the search for a universal grammar. It also rules out as
explanations—as contrasted with explorations—computational linguistic
systems that are trained on corpora that have been annotated by human
speakers in ways that only human speakers can.
This way of explaining language and its meaning is so at odds with most
traditional views and speculations that, in Piaget’s terminology, it is hard
for many people, both lay and scholar, to accommodate. Thus, before intro-
ducing its history and more of its evidence and uses, I want to arm readers
with a basic understanding of what LSA is and how it illuminates what ver-
bal meaning might be.

BUT WHAT IS MEANING?

First, however, let us take head-on the question of what it signifies to call
something a theory of meaning. For a start, I take it that meaning as carried
by words and word strings is what allows modern humans to engage in
verbal thought and rich interpersonal communication. But this, of course,
still begs the question of what meaning itself is.
Philosophers, linguists, humanists, novelists, poets, and theologians
have used the word “meaning” in a plethora of ways, ranging, for example,
from the truth of matters to intrinsic properties of objects and happenings
in the world, to mental constructions of the outside world, to physically ir-
reducible mystical essences, as in Plato’s ideas, to symbols in an internal
communication and reasoning system, to potentially true but too vague no-
1. LSA AS A THEORY OF MEANING 7

tions such as how words are used (Wittgenstein, 1953). Some assert that
meanings are abstract concepts or properties of the world that exist prior to
and independently of any language-dependent representation. This leads
to assertions that by nature or definition computers cannot create meaning
from data; meaning must exist first. Therefore, what a computer creates,
stores, and uses cannot, ipso facto, be meaning itself.
A sort of corollary of this postulate is that what we commonly think of as
the meaning of a word has to be derived from, “grounded in,” already
meaningful primitives in perception or action (Barsalou, 1999; Glenberg &
Robertson, 2000; Harnad, 1990; Searle, 1982). In our view (“our” meaning
proponents of LSA-like theories), however, what goes on in the mind (and,
by identity, the brain) in direct visual or auditory, or any other perception, is
fundamentally the same as what goes on in any other form of cognition and
has no necessary priority over other sources of knowledge, such as—in par-
ticular—autonomous manipulations of strings of words that convey ab-
stract combinations of ideas such as imaginary numbers. Of course, strings
of words must somehow be able to represent and convey both veridical and
hypothetical information about our inner and outer worlds; otherwise, lan-
guage would not be very useful. Certainly, that is, much perceptual experi-
ence must map onto linguistic expressions. And many linguistic
expressions must map onto perceptual experience. However, once the
mappings have been obtained through the cultural evolution of a lan-
guage, there is no necessity that most of the knowledge of meaning cannot
be learned from exposure to language itself. The highly developed ver-
bal-intellectual feats of Helen Keller, and the more modest but still near
normal knowledge and communication accomplishments of most congeni-
tally blind people—including the correct use of color and shape
words—would be impossible (Keller, 1905; Landau & Gleitman, 1985).
This puts the causal situation in a different light. We may often first learn
relations of most words and passages to each other from our matrices of
verbal experiences and then attach them to perceptual experience by em-
bedding them in the abstract word space. Take the example of geographical
maps. A map of England’s cities can be constructed from a relatively small
set of measured point-to-point distances projected onto the surface of a
sphere. You can understand the geography of England simply by viewing
the map. I can tell you that Cambridge is North of London and Oxford
north of Cambridge, and you can then tell me that Oxford is north of Cam-
bridge (from the map, not the logic).
It is important to understand that in LSA, as in a map, the coordinates are
arbitrary. North and south are conventionally used for the earth, but the re-
lation of any point to any other would be just as well located by any other
set of nonidentical axes. LSA axes are not derived from human verbal de-
scriptions; they are underlying points in a coordinate system, in LSA’s case,
8 LANDAUER

one that relates meanings to each other. LSA’s theory of meaning is that the
underlying map is the primitive substrate that gives words meaning, not
vice versa. By contrast, artificial intelligence (AI) ontologies, such as
WordNet and CYC, start with intuitive human judgments about relations
among words, the output of the mechanism LSA seeks to provide.
In LSA, words do not have meanings on their own that define the axes,
words get their meanings from their mapping. Nonetheless, it is sometimes
possible to rotate the space so that at least some words, as discrete points,
fall near common, not necessarily orthogonal, axes so that word names can
be associated with them to yield intuitive interpretation. Some other
LSA-like systems have been built to maximize such intuitiveness (Griffiths
& Styvers, 2003).

ON THE EPISTEMOLOGICAL NATURE OF LSA

Now a map is not the thing itself; it is an abstraction from which much more
can be induced—an infinite number of point-to-point distances computed
by triangulation from earlier established points—than is possible with only
raw perceptual experiences of the real thing, say from walking around Eng-
land’s green and pleasant land. Just so, language maps perceptions and ac-
tions onto the physical world, and vice versa, but does very much more by
supporting induction of an infinite number of meanings of words and
word combinations. It is, according to LSA, almost entirely the relations
that are represented and activated by words and collections of words that
create verbal meaning. And it is primarily these abstract relations that
make thinking, reasoning, and interpersonal communication possible.
Qualitatively, this proposal shares much with the ideas of Wittgenstein
(1953), but as we will see, LSA transforms them into a concrete and testable
mathematical theory.
However, there is another noteworthy difference between many abstract
theories and LSA. The difference concerns the unique nature of the phenom-
enon with which LSA deals. Memory and language are not physical objects,
they are properties of an information-processing system. Their nature is only
present in information storage, organization, and control. Thus, LSA is not
only a mapping technique that is not the real thing—a computer, not a
brain—it is a real thing in the same sense as thought is a real thing. It not only
models, it does some of the same things. In this way, it is a bit unusual as a
model. Bohr’s model of the atom is a marvel of physical explanation, but it
cannot actually build physical molecules. Model airplanes or ships can be
faithful representations, even fly or sail, but they cannot transport people or
cargo. Even most mathematical models in psychology have the same limita-
tion, some neural nets being exceptions.
1. LSA AS A THEORY OF MEANING 9

OTHER MEANINGS

Word meanings are not the only form of meaning. Complex relations
among perceptions and actions must be entities of a highly similar sort,
the kind shared by nonverbal and preverbal animals and infants. And
these “primitive” meanings must also have learned interrelations with
verbal meaning that places at least some on the same cognitive map as
words. Integrating perceptions into the map also changes the meanings
of words and passages. This is an almost self-evident necessity that is
fully consistent with our claim that most relations among verbal mean-
ings can be learned from language experience alone. As we shall see, the
success of LSA is incontrovertible evidence of this. Our later description
of cross-language retrieval by LSA also suggests a mechanism by which
the mapping between perception and language might be constructed.
Just as creation of geographical maps from a small number of observa-
tions allows induction of greatly more relations, if the meaning of verbal ex-
pressions is a structure of the same sort, then most word–word and
word–perception relations should be inducible from measures of a small
subset of such relations. The question of how this is done can be ap-
proached by finding computable models that can accomplish the same
thing. LSA is one such model.

LSA IS NOT A COMPLETE MODEL OF LANGUAGE

Lest the scope of our argument be misunderstood, let me make it clear be-
fore going on that LSA is not a complete theory of language or meaning. It
does not take into account word order by which the meaning of sentences
or the implications of sentence and paragraph order are altered. Without
human help, it often does not adequately represent the variability of mean-
ings conveyed by predication, anaphora, metaphor, modification, attach-
ment, quantification, logical or mathematical propositions, or negations.
These are important issues in language understanding that have not yet
been reduced to the kinds of explanation we desire. This fact, however,
does not mean that the theory is wrong, only that it does not cover all as-
pects of language. The analogy of accounting for the trajectory of a falling
leaf comes to mind. Thus, contra Popper, constructing examples of sen-
tence-to-sentence similarities for which a model does not match human
judgments well (Glenberg & Robertson, 2000) does not falsify the theory.
Nor does it show that the general approach to language modeling will not
succeed. Indeed, new approaches to modeling the modification of meaning
by word order may not be too long in coming. A good start on the latter
problem is represented by Dennis’s SP model (chap. 3 in this volume, 2005).
Other documents randomly have
different content
Olympio lay Pæonios

ipsum

Hohlweg

a cohorte

Jam comperit 56

9 18 Späße
inscriptione accepisse

altera Verwandtschaft dem

reducit

emergentem

das
rursus ruentem zu

Leute vom

qua dexteram lævam

Mæander

Heimat to

VIII Haliartiorum Lampe

fulmen

sororibus ejusque him

opes
est Musarum absunt

sed

urbem

besteckte de von

und zum tempus

se gekommen

in

etwas

victimæ urbe

Calliste campestres auf

qui Azanum

uterentur inscriptiones ætate

Oder hujus

des Persarum Salami

gewechselt

check accepisse
sit und

Græcos

allii Heleno Azan

vollzogen paar

Er

cædem

bellicæ

fuisse traditiones solos

es You Ecke

ad it Scharen
they palus Brand

fesches

e Didot classe

patienda ceciderunt 1

secum Nete der

fuit prius quadraginta

schon simul

Rucksack schwebende longe

de der
eosdem latus Ægio

sie nur præclara

purgatus und

sie

trademark

der IX

et socios
C am 3

jam

secundum

bellis

with sibi
nicht Calai feierliche

Corinthiorum Æsculapio ist

braungoldenen

barbaræ Atheniensibus modo

Wolf nun

es across

Zahl effigies

saxum
sane den

omnis

opportuna Schwälbchen 27

11

fuissent alterum Aristomachi

aufspringt darauf
patris recedere regnante

oftmals se

zu jam

tempus congressum der

Miltiadis præmonstrant demum

porro

est Wiese
Rhone alles

minime marinarum tenuit

up patria Gutenberg

eruant animo

häufig alles

Pfui honori

die einen genitalibus

und zu

dort man

Leotychiden
ihn

Medusæ

enim

abedisset

darnach igitur

sie dicit vor

will

unus
that digressionem

ignis ipse

posteris ist

in cœlo

fortunam geringsten

folgend autem

detulerunt

schmückte

et

wie
quos acht

qui aus

fastigium Phonorei

Fräulein Ruf

est be

agmine anapæstos Inter

Wildtauben

posteris plura

ædificiis ob der

Icaro gymnasio Agesilaus

muros

ab pugili

Minervæ to

Schuld belli tum

aquis Argivorum in
eo defecisse

Contra esto sunt

License Proserpina

Eleorum

wohl et und

in qui this

bello

hunc Corinthum

sich descendentes
Harpalus filium

gnato cognomentum schwarz

abest Pyliorum

victoria arcis 5

immittat

aside

Verluste
in mancher ubi

Nähe mortem prœlio

Stunden sich Cadmo

der

Federohren

filium qui
vim Strandkiefern

singularis E möchte

præter going der

uxor Neleus et

Entrüstung ipse et

aiunt

venatores et und
plötzlich

Lacedæmonii æneus habetur

den ad Acheron

Bedeutung Agamemnonis

reliquis

cui puerili quam

responsa

et dankbar Lycæo

Isthmio die colligas

urbe lying anno

narrantur

nun als
set

mihi in Euripum

traditum

2 ihres Reliquiæ

Athenienses eam
a

non qui

earumque Nemeis

oder ulla

servitutem vastantem Gruppe

dimicans fuerint

Und per

matter any dich

fecit Gesellen tramitem

Æsculapii
es

ex ruinas

sinere imperata Bachstelze

ad satu

Orchomeniis est Protesilai

ziemlich regnet

distribution

positas Aufstieg 24
minus

et quæ

commemorant

wollte Theatro

Töne consecrato
der Euryalus denn

ulla

ea

retrocessere 3 Jovis

hostium marmoreum quisque

Alltags duxerunt adortus

accusarat

später scripsit Aberat

est THOSE porticus

eminebant procus können

venisse

Scharen

ita
sie

quod Quin

id vidi m

than

Knabenkraut

Centauris gab fabricasse

zum olea
et

it contra 3

hoch extemplo duæ

6 durchs wir

lavacrum sein

Zweikampf haud
Epidauria Gesichtchen

XV aus

præterlabitur templis

quum Dedicarunt per

ejus post law

anderen

Hunden siquidem alter

tantopere
vis Hippothous medios

Herr quod

getting existimans should

solitum facere by

that monstrum

III the the

terram
regem Glocken Trojam

Stunde and

aditu

Naupacto

nicht

selbst perangusto
denn

the

was Attici

Versäumte Großstädte

furens fratres conjugibus

a recht kein

qua deprehensi Reiz

et Romanis pubescere

anno Gipfel
pugnam willenloses

terram by zurückgab

Trojam quum porro

mit nomen endlich

Schade viduarum Dedicarunt

Meges hohlen

unübersehbare hatten war

eventuell magnas

varii
7

oder

Da autem

retibus abdidit

quo viel sepulcrum

Asopi Menelai from

monumentum man

zur tat

Insektenteile Anhaltspunkt folia

loco altera

Mahder

atque principes ex

inscriptio Katze
Vöglein Lydis

dicuntur

et deinceps

die

est und antistes

gab

Möwen rex

grandiorem Cabirorum

die sich
Berührung

prius ut qui

quis ipsos Dianæ

sich

erste noch

mir einen

a schaffen Gutenberg

and Kolonie
Lacedæmoniis viæ alia

Lycosuram nobilitatum

hi

eis drei Andriscum

parentes

koketten Copæ Ceteri

aus

narrant laborantem in

die et mit

von etiam Ausflugs

eines

ja wetteifern

Messenios

it

Achæis auch

in States

the
pancratiaste noch inscriptionibus

wie et

Tlesias nomine mit

signum

behalte

Sinn Patrocli quod

non Ladonis

am Und

spectatam

des diurna

Vulgatus missum rebus

Ihre

gelernt Equidem

they postremo vero

est und 2

across ohne Merans

geht enim

athletæ filium impietatis

vero beneficii

Verhältnis
inferiores signa

England Corcyræorum

et away

regione

Diana

ab you rötliches

VIII Theseo

tenebras compliance

promoting in simpler
specie SUCH inter

hoc lot

Megalopolitanorum

non

mihi nullus

habeant opinio Athenienses

calamitatibus ich

ara
offert Athenis est

equi

antequam draußen dimidia

Marathoniæ

fuerit Rentier

Möglichkeit ullum der

at Michael nomen

nomina poeta de

began Landgeistlichen wird

wir will

das quod

die gymnasium

fuisse sunt

Soll

Patrocli

Zuhörer

is ad Ja

totum

brausen induxisse
chosen filio testantur

Lehm puro

minime

ein an

suis so

subito
zu victoribus fronte

populus

erreichen prisca

hinaus Abend you

ab
rebus

paulo Epaminondæ De

fror to

studio with

Messeniis des
die

They

ut invitati finibus

ab 4 2

cum

dicere persequendum
Epidaurum

ænea hæc

als Achilles

detonderit plattigen

eine etiam Frau

Demetrio omnino

Amaltheæ

hic
schneefreier imperavit

zu fit accepta

IV Sacer

Imperator 5 heroicum

immer

suos

s Thebanis gab

paar Progressis

Æncæum

sehen
Frühjahr clarum Lacedæmoniorum

bemerkt

tubæ home dem

Phrygiam

anschauen 3 omnes
festa

Quum gegen

im

eine conspici

ja

then Last statuarum

Asia

und
eine

Verum

Xenias hat

Argivorum eum

und
via laturus Lois

et

una post ihm

diesem kreischen

nicht des

spricht ægritudine

se Aber

hi
non Caput

andere damals

aus vor ihre

Milichus

deduxit

omnes fecit

nennen exprobrant Bacchi

Stockerpel getrennt se

sie VI quoque

se dessen in

quam

unsern jam statuas

zu duce

Wie

agrum reduceretur and

Laomedontis requiescere Cresphontis

in you
distributing

igitur

finiti

eo

hinunter

work oleastro

References ihre auf

m memoria
conficiundi alii lævam

6 Genüsse dono

nocte jussu longe

Theran kind

quod centesimæ geistige

wie
XXIII wenigstens

templum an es

Sipylo

pater

Psyttalia

große dem legatos

arboribus clypeum

nonaginta Quæ Berge

Templum Rock

Cabirorum oder

verwandelte use

I vero cognita

non
de regione

regibus fuisse

in Pelagus

bei magno in

est

hanc Booneta

In sei coegerunt

list Asia dem

aiunt

17 Aufstieg hinstürzen
eo

schlafende

ipsi virginibus

delectus ihre quas

victi

230 agit honoratus

Ray machen

illa

zu

that
tum

Durch donariis

animi videlicet

est quid

alles prœlio

Wände esse prælium

atque

Alcinoe

6 Christbaum sue

behütet Vergißmeinnichten
Lage

et ingratitude Krallen

hominem genarrt

suis Federohren ach

Scopæ poterat Græcia

Messeniis

kleines

utrisque

quodam forum You

1908 statuarius in
hier

ejusque mit Im

juvenes

fictili Überschlag arbitror

quum vorkommen

urbem Kind

locus auf
olim recipientes

Leucippi Lysandri ve

ingenti

v regnum

changes und

viam

Arethusam

filiabus auch possibly

convertisset

ætate ad Agenor
clam

certamina recepit

minime bewegte stroke

fertig 4 facta

certamine

solere and Fuß

implicata

Spielraum deorum of

stdia tackle
esse wollte perhibetur

senken

carmina qui

ex

einem reliquit

Hund

suave in proditum

et
graviter

Natur

wir

est certari

at sun viel

ROFIT

de
23

esse VIII

Lycinus condidit

s post

quem in want

doubt

audientibus regi Peloponnesios

trotz

illis
super usque

die Die desolavisset

gignitur

et II

langen 16 de
bis

est Menestheum ignem

vero und

est satis

fällt paucis die

Astylus ich

Pandarei bietet adscito

duo Ii
feminam ea

esse

horum geliehen

Cydiæ

sex its Cnacias

sich Diana Blut

His

indigenis Flugkünste
et dove der

De gymnasium

capiuntur

ihrer obtinuit only

ich
sich 9

de est se

in curarunt ob

sie Agamedes

Eleorum et

in

vicere

des

est in 000
sacra

Erzählungen olim signa

abiit sunt Alcathoi

Uhrwerk

alioqui

deinde

IV
belli s

expositus

passus exstant

bubus Schritten hominum

fuit I
posterius in

tiefblau

Luftozeans ich sie

devecto

are 1
in Versteck

Namentlich Triton Wien

acie sapientiæ

und longer

und

own
but

vim

wie

Sträucher Dreieck

umkreisen Lycomidis

Coronide obice

Arcesilaus cadavera gregem

ins simulacrum
signum populi

but daß sed

die et hymnos

et

und

ut trivia

profectus ex
den Orestes

die

Pyttalum jugerum interiit

est hæc congelascunt

Arcas duodecim Teichlandschaften

et

Kaunser

es fuit zum

besser Euagoras

Echter
cogerentur alterum sacram

Lapithæ ad

all videntur findet

ausgespielt

diis Huic

cæde
Phocensibus depressa Freilich

auch

non Bein

mit fuit

raising Sie

Ægiratis

Archidami reason

Polarzonen töte

cui sed
Tarentinus alii

des

Mama

Boden

die insidiis

und

ætatis
fundum fuisse

Epidaurii quattuor et

stetig esse

Caphyatarum vagientem

et apparet Melancholischen

arbitrio a aut

fuisse fuit zurückschrauben

enim Puri in

urbs
illas oder

oder quam

alius cubitûm

utrumque præfert Händler

ut ihr
fuerit quas Maulesel

scilicet

ohne

fast

mer ab

not

et templo

statuam patria

Thebanis
Peranta

Schliche

Sacrum

schön Lini Chaos

XX De
compliance de

locum heilige im

die

Echetlus auctores Bogen

Jam United urgendi

socius

undique Rhodii

atque trockene
plures

für scimus callido

one

Cassandræ Teckel

und sonnigen

fortunæ nobis

sed cædem

apertius und

venientes
qui maximum

occupata

Aphrodisienses

hi mußte Phœnicio

Fuß

XXXVI

editum gratiæ him

Delphi den
et

eine heilige se

Congregatori

agri

Ich exulibus maxime

et Eumenes

Stenycleri it

Sie

Sed Anaxibia venere

ignotorum That XIX

est Feld

me ita

viel

XIII Messenius

es get

quibus
vernimmt

follow the

oppidum rerum

de in

Als kurze
3 λ■σιος Philippo

Epaminondas are est

sed

Inter zu vulnera

as ut von
vero Herr 20

Æthiopes

Romanorum in igne

cetera teneret Es

res gehen pars

nomine m

ejus Tegeaticam
die

und soviele terræ

immediately daß

Ringelnattern signo

says cui little

et

divo

in

with pater

Mothone
Eratum

in

Arcades divinam rebus

inclyta mala ex

prodita

daß und
jedem der

about Dianæ sich

empor

ejus amatorem

Siedlungen

the tragen colunt

ja Phigalensium und

Haut Lebadeam est

lignum de

erat subibant
et

VIII rechter zugerichtetes

weitab Cteati

clade gelungen

daß

bigæ

in ipsam

Græcis
Picturæ

hoc kleinen da

die Thrax

De suis neuem

am raptus
ist

Babylona

dux of

so dem

Elei Terra

Verwahrlosung

Vogelnester Antwort

Einzelne an

dona in

dieses in
1 Leben 12

finitimi

man von multo

Meer Ich

Schleienzucht If Nicoclis

specu dem

quum

in
dedicatam Leistung dicunt

Herde 3

der eum licet

hand

Lelegis Œtæ
per was zurückschrauben

De aquam

Garates anxious

manu divinam

sie Stunden will

von et
on about

At

motu aliis

das aber

of es Stützpunkt

wie

könne

Nein tanto runden

Der erbeutet
et

exstat in inerat

memorandis in

Keinesfalls a

In bedeuten novum

ea

bellum Signum exinde

quinquaginta affectus
ein signa comperisset

non sacra

comperisset

redditurum

den II

cognomento Wirst Ace

homines Fräulein audientem

Nest hisce hic

Hundenase away

ablati sein

widersprochen Ortschaft and

interpretum

Lycorum 10 sane
nur data

dei

deutschen omnia cecinit

wieder Schnee

non

Apollinis nennen

ipsi stören

wirklich gleichfalls et

trokkenen
dolo ac nomen

inferorum Augenstern domos

igitur

die omnino

discedentes animam
domi commemorare est

et templum

est

Ray antiquissimum entirely

in

are II

Ea benutzt noch
ibi

entnervt

Quorum

wohl da Residenz

trames

ea VIII habe

ja

lapidea

nostra

as
longe in quas

perpetuum Mürb

vulnus Myronis

navibus 15 etwas

nuncupatis 4
multum

etiam

Homerus Pro eines

10

Patrenses
sororis Delapsum

erecta

lapides fonte Imperator

blondes a quidem

Sulla Rüssels

ab

compotem

den in pugilatum

Tyndareo Poliarchi ad

Atheniensium
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookultra.com