288 CIMINO, From Data to Knowledge

Review n

From Data to Knowledge

through Concept-oriented
Terminologies:
Experience with the Medical
Entities Dictionary

JAMES J. CIMINO, MD

Abstract Knowledge representation involves enumeration of conceptual symbols and

arrangement of these symbols into some meaningful structure. Medical knowledge representation
has traditionally focused more on the structure than the symbols. Several significant efforts are
under way, at local, national, and international levels, to address the representation of the
symbols though the creation of high-quality terminologies that are themselves knowledge based.
This paper reviews these efforts, including the Medical Entities Dictionary (MED) in use at
Columbia University and the New York Presbyterian Hospital. A decade’s experience with the
MED is summarized to serve as a proof-of-concept that knowledge-based terminologies can
support the use of coded patient data for a variety of knowledge-based activities, including the
improved understanding of patient data, the access of information sources relevant to specific
patient care problems, the application of expert systems directly to the care of patients, and the
discovery of new medical knowledge. The terminological knowledge in the MED has also been
used successfully to support clinical application development and maintenance, including that of
the MED itself. On the basis of this experience, current efforts to create standard knowledge-
based terminologies appear to be justified.
n J Am Med Inform Assoc. 2000;7:288–297.

The first step on the path to knowledge is getting things our ability to care for specific patients and learn more
by their right names. —CHINESE SAYING about biomedicine in general.
A basic tenet of medical informatics is that if com- One approach to such representation is knowledge rep-
puters are to help us with the process of health care, resentation, a collection of techniques drawn from
they must be able to manipulate information symbol- computer science. There are many such techniques,
ically rather than simply store and regurgitate it. If but they all share a common approach of using sym-
we can represent data about the patient and knowl- bols (usually represented with terms from a controlled
edge about health care appropriately, our computer terminology) and structures for arranging the sym-
systems can accomplish many tasks that will enhance bols. In this paper, I review some of these techniques
and examine how medical informaticians are apply-
ing them to the task of representing knowledge about
the symbols (that is, the terminologies) themselves. I
Affiliation of the author: Columbia University College of Phy- illustrate the advantages of this approach with ex-
sicians and Surgeons, New York, New York.
amples drawn from the work of my colleagues and
This paper is based on a presentation by Dr. Cimino that was myself at Columbia University, to show how a knowl-
part of the Cornerstone on Representing Knowledge, one of edge-based terminology can help us get raw patient
four Cornerstone sessions included in the program of the AMIA
Annual Symposium, Washington, D.C., Nov. 6–8, 1999.
data ‘‘by the right names’’ and set us on the path to
knowledge, to:
Correspondence and reprints: James J. Cimino, MD, Columbia-
Presbyterian Medical Center, Atchley Pavilion, Room 1310, 161 n Gain a better understanding of our patients
Fort Washington Avenue, New York, NY 10032;
e-mail: ^[email protected]&. n Access knowledge relevant to the care of our pa-
tients
Received for publication: 11/15/99; accepted for publication:
11/18/99. n Enable the use of smart systems to apply knowl-
Journal of the American Medical Informatics Association Volume 7 Number 3 May / Jun 2000 289

edge to the care of our patients create them for each application. These terminologies
offered little in the way for formal representation, be-
n Discover new knowledge from health data
yond simple strict hierarchies. Eventually, these
schemes were found to be inadequate, and informat-
Such knowledge can also be used, it turns out, to ics researchers began seeking ways to use knowledge
manage complex clinical applications, including the to represent the terminologies themselves in order to
maintenance of the terminological knowledge itself.* support better comprehension, use, and maintenance.1

Knowledge Representation in Medicine Like other knowledge representation tasks, the choice
of formats for terminological knowledge differed from
Representation of medical knowledge was one of the application to application. My colleagues and I1,8
first tasks addressed at the advent of medical infor- chose a frame-based representation for terminology,
matics, starting with Ledley and Lusted’s landmark as did Masarie et al.9 Bernauer10,11 used an object-ori-
paper2 describing the use of punch cards for indicat- ented approach expressed with conceptual graphs.
ing relationships between diseases and their manifes- These two approaches, shown in Figure 1, and their
tations. Since then, informaticians have drawn on variations have become the most widely used repre-
computer science for a variety of techniques. Occa- sentation schemes.
sionally the influence has flowed in the opposite di-
rection, as with Shortliffe and colleagues’ MYCIN Over the past decade, knowledge-based representa-
project.3 A full review of knowledge representation tion of terminologies has accelerated. These tech-
methods is beyond the scope of this paper; however, niques have been applied to existing terminologies in
one comparative study will serve to illustrate some of order to make them more understandable and, hence,
the general principles. usable. Campbell and Musen12 demonstrated that the
Systematized Nomenclature of Medicine (SNOMED)
Starren and Xie4 examined a guideline for cholesterol could be represented using conceptual graphs in a
management and represented it using three different way that offered the potential for more consistent
formalisms: PROLOG (a first-order logic-based sys- SNOMED coding. This theoretic approach has been ap-
tem), CLASSIC (a frame-based system), and CLIPS (a plied to a large project to expand SNOMED content
production rule-based system). The authors con- with the Convergent Medical Terminology (CMT)
cluded that ‘‘all three representations proved ade- project between Kaiser Permanente and the Mayo
quate for encoding the guideline.’’ Despite the differ- Clinic.13 More recently, Spackman et al.14 have de-
ences in notation, the underlying symbols used in the scribed significant efforts by the College of American
schemes were essentially the same. This suggests that Pathologists to represent SNOMED terms with logic-
while the structure chosen for representing knowl- based descriptions. Bakken et al.15 have used a similar
edge may be important for practical considerations approach to represent several nursing terminologies.
such as execution efficiency, the real heart of the
knowledge lies in the symbols themselves. In fact, van In contrast, some researchers have addressed the
der Lei and Musen5 have argued that typical rule- knowledge representation issue before creating actual
based systems do not encode true medical knowledge. content. Rector et al.16 have undertaken the GALEN
project to provide a foundation for representing ter-
Knowledge-based Terminology Representation minologies that can span the multiple languages en-
compassed by the European Community. Using a rep-
Knowledge-based systems, and medical computing resentation language called GRAIL, they have
systems in general, have traditionally treated the developed a coding reference (CORE) model for de-
coded terms they use as simple placeholders for con- fining ways of assembling medical terms. A number
cepts that are understood by the users of the systems of experiments are under way to test the usefulness
but not by the systems themselves.6,7 As systems have of their formalisms. For example, Brown et al.17 have
become more sophisticated, their terminology needs described the efforts of the National Health Service in
have grown. At first, it was sufficient to turn to large, the United Kingdom to represent definitional knowl-
existing, standard terminologies to avoid the need to edge of the Read Codes using the GALEN model.
Hardiker and Rector18 have also used GRAIL to rep-
*Knowledge-based systems typically reason about some part of
resent terms from nursing terminologies.
the world outside their knowledge base but not about the in-
formation contained in their knowledge bases; that is, they are As new, special-purpose terminologies have arisen,
usually not introspective.1 their creators have begun turning to knowledge-based
290 CIMINO, From Data to Knowledge

part of the Gálapagos tools set by Campbell et al.13 for

use on the CMT (convergent medical terminology)
project.
No description of terminological efforts would be
complete without inclusion of the Unified Medical
Language System (UMLS). Originally envisioned as a
way ‘‘to improve the ability of computer programs to
‘understand’ the biomedical meaning in user inquiries
and to use this understanding to retrieve and inte-
grate relevant machine-readable information for
users,’’ 26 it has initially been focused on the more
F i g u r e 1 Two representations of the medical concept modest goal of supporting ‘‘the development of user-
‘‘Serum Glucose Test,’’ using frame-based (top) and con-
ceptual graph (bottom) formalisms. In each case, the other
friendly systems that can effectively retrieve and in-
terms (‘‘Laboratory Test,’’ ‘‘Serum Specimen,’’ and ‘‘Glu- tegrate relevant information from disparate machine-
cose’’) are also controlled terms represented with their readable sources.’’ 27 The UMLS provides a knowledge
own knowledge. base not about the meanings of terms, per se, but
about models used by existing source terminologies
representations. The Logical Observations, Identifiers, and how they relate to one another. So, for example,
Names, and Codes (LOINC) project, described by the information the UMLS provides about a labora-
Huff et al.,19 started with a formal representation of tory test term will include which source terminologies
the definitions of laboratory tests, using an approach it comes from, which terms it is related to in the hi-
that is similar to (but much richer than) the examples erarchies of those source terminologies, what its syn-
given in Figure 1. This approach has facilitated the onyms and lexical forms are, and which other terms
adoption and use of LOINC across multiple institu- it is related to in some source terminology. It does not,
tions.20,21 In the domain of drug terminologies, used however, strive to provide definitional information
by pharmacy systems, commercial efforts have fo- (such as what the test measures are or what its spec-
cused on representing knowledge about pharmaceu- imen is) unless that information is available from a
tical products that includes definitional information source terminology.
about ingredients and formulation (T. McNamara, C.
Broverman, K. Eckert, M. Moore: personal communi- The Medical Entities Dictionary
cations, 1998, 1999).
The creation of terminological knowledge bases has The knowledge-based terminology effort at Columbia
led to development of knowledge-based tools for sup- University and the New York Presbyterian Hospital1
porting their development and use. A vocabulary has grown into a repository called the Medical Enti-
server called VOSER has been described by Rocha et ties Dictionary (MED).28 It currently contains some
al.22 for use at the LDS Hospital, Rector et al.23 have 60,000 concepts organized into a semantic network of
described the GALEN server, and Chute et al.24 have frame-based term descriptions. Terms are drawn from
recently enumerated the minimum desirable charac- those used in laboratory, pharmacy, radiology, and
teristics for vocabulary servers. Knowledge-based ed- billing systems. It includes 208,000 synonyms, 84,000
iting tools have been developed for terminology con- hierarchic relations, 114,000 other semantic relations,
struction by Mays et al.25 and have been adapted as and 66,000 mappings to other terminologies, includ-

F i g u r e 2 Example from the Medical Entities

Dictionary. The term in the box (Plasma Glucose
Test) is shown in relation to its parents in the is-
a hierarchy (solid lines) and by nonhierarchic se-
mantic links (broken lines) to other terms in the
network.
Journal of the American Medical Informatics Association Volume 7 Number 3 May / Jun 2000 291

ing the UMLS and LOINC. The relationships in the spreadsheets to the MED could be handled without
network provide definitional knowledge about the in- modification to the program. When a new Web-based
dividual terms. For example, laboratory test terms are application (called WebCIS) was developed to replace
linked (via ‘‘substance-measured’’ relationships) to DOP, the same knowledge in the MED was reused to
chemicals they measure, medication terms are linked support display of laboratory test results.31 Figure 3
(via ‘‘has-ingredient’’ relationships) to their chemical shows sample displays from both applications.
ingredients, and diseases terms are related (via ‘‘has-
While the use of the MED knowledge was automated
location’’ relationships) to their body locations. Figure
and dynamic, its maintenance was manual and tedi-
2 provides some examples of this knowledge.
ous. Elhanan,32 who was charged with this duty,
The MED was constructed to serve the primary pur- viewed the task as a knowledge engineering problem
pose of a repository for codes and terms used by clin- and sought to find ways to use the knowledge to sup-
ical applications to represent data in the clinical data port the acquisition of new knowledge about the
repository.29 The knowledge included in the MED was problem domain (i.e., the relations between test terms
originally intended to support intelligent vocabulary and spreadsheets). The result was an expert system
management tools. However, as the repository grew that could be used to automatically audit the appli-
and the data in it were reused in a variety of ways, cation knowledge in the MED, support its mainte-
the MED knowledge was reused as well. In many nance, and ultimately drive the performance of the
cases, the MED served as a convenient repository for clinical applications. It would, for example, identify
additional knowledge used by various applications, tests that could not be displayed by any spreadsheet
and so it grew to serve as a tight link between clinical and make suggestions about how to link them to ex-
applications and the terminologies used by them. isting spreadsheets.32

Application of Knowledge–based Smarter Retrievals from the Record

Terminology: Proof of Concepts Specific knowledge about patients is crucial to their
care. Although the aggregation of data in the clinical
Over the years, application developers, researchers, record holds much of this knowledge, the amount and
and students have shown great creativity in exploiting organization of the data can render the knowledge
the MED model and its content. For this paper, I have obscure. Because the MED contains knowledge about
collected their work and attempted to summarize the how data are coded in the record, Zeng33 sought to
kinds of roles the MED has played in bridging be- supplement the MED with knowledge about how
tween the data encoded with its terms and the ad- these data might be aggregated into concept-oriented
vancement of some aspects of human knowledge. views of the medical record—for example, with re-
Much of this work is anecdotal, so far as the MED is spect to a particular patient problem. She was able to
concerned; there are undoubtedly other terminologi- extract information from other existing knowledge ba-
cal models that could have supported the various ses and reuse it in the MED. From this information,
projects described here. However, taken in aggregate, she was able to generate queries automatically to ex-
I believe they provide substantial evidence that tract problem-specific data from the record. She was
knowledge-based terminologies have great potential then able to assemble them into views that were de-
for furthering the goals of medical informatics. monstrably better than the more traditional time-ori-
ented views. For example, if a user specifies interest
Merging Data and Application Knowledge in the problem ‘‘Pulmonary Heart Disease,’’ the ap-
plication will identify test results that measure rele-
Knowledge about the operation of clinical applica- vant substances (such as oxygen and carbon dioxide),
tions may be stated in written documentation, but is reports on examinations of relevant body parts (such
only occasionally described using formal modeling as cardiograms and chest x-rays), and medication or-
tools. Although the MED was not intended for appli- ders (Figure 4) that are relevant to the treatment of
cation modeling, developers of the Decision-sup- the condition.
ported Outpatient Practice (DOP) application found it
useful to include the various laboratory data spread-
‘‘Just-in-Time’’ Education
sheets as concepts in the MED.30 Because each spread-
sheet was linked in the MED to the appropriate test When an information need arises during the course
classes (each of which corresponded to a row in the of caring for a patient, an opportunity arises to supply
spreadsheet), DOP was able to display test results dy- specific knowledge to meet that need and, in the pro-
namically, such that the addition of new tests and cess, educate the clinician. Referred to as ‘‘just-in-time
292 CIMINO, From Data to Knowledge

F i g u r e 3 Screens from two applications that use the Medical Entities Dictionary (MED) knowledge about spread-
sheets. Top, A display from the Decision-supported Outpatient Practice application, an X-Window application, showing
the Chem-20 spreadsheet. Each row corresponds to a class of laboratory test terms in the MED. Bottom, Use of the
same information by WebCIS to create a Chem-20 display for the Web.

education’’ (G. O. Barnett, personal communication, have used the MED to support such tasks through a
1997), computer systems can assist with this task if variety of applications that we refer to collectively as
they have sufficient knowledge about the context of ‘‘infobuttons.’’
care to anticipate the information need and if they
have enough information about potential resources to The first such application used the MED, together
help direct the clinician. They can also facilitate re- with the UMLS, to provide translations from diag-
trieval of the specific relevant information by using nosis and procedure codes in a patient’s record to
data about the patient to seed the search strategy. We MeSH terms for searching the medical literature.34 Al-
Journal of the American Medical Informatics Association Volume 7 Number 3 May / Jun 2000 293

F i g u r e 4 A concept-ori-
ented view of a patient’s
medical record, generated
from Medical Entities Dic-
tionary knowledge. In this
example, the problem of in-
terest is pulmonary heart
disease, and the data dis-
played are a subset of med-
ication orders.

though the application did manage to carry out au- bear on specific patient problems when relevant data
tomatic MEDLINE searches, the technical process was are supplied to the system. Elhanan et al.39 used the
awkward and unreliable. The advent of the World MED to convert laboratory results into findings rec-
Wide Web has greatly simplified our ability to inte- ognized by a diagnostic expert system called
grate online resources with our clinical information DXplain.40 The terms were converted by linking test
system. As a result, several different infobuttons have terms (such as ‘‘Serum Sodium Test’’) to measurable
been created to link coded data35 and text reports36 to substances (such as ‘‘Sodium’’), which were, in turn,
relevant resources such as PubMed, pharmacology linked to findings (such as ‘‘Hyponatremia’’ and ‘‘Hy-
reference books, and library materials in Utah, Wis- pernatremia’’). In this way, a panel of test results
consin, and England. could be converted to a patient description and
passed to DXplain to obtain a differential diagnosis
(Figure 5).
Expert Systems
Automated guidelines are another form of expert sys-
A centerpiece of our clinical information system has tem that has been successfully integrated with our
been the clinical decision support system.37 The MED clinical systems. Applications that encode the guide-
supports this system by integrating the relatively lines for cholesterol management41 and mammogra-
high-level terms used in decision rules (such as phy recommendations42 have been integrated into the
‘‘Blood Sugar’’) with the relatively low-level terms PatCIS (Patient Clinical Information System) project.43
used in the clinical record (such as ‘‘Stat Whole Blood Users of the system can have their data automatically
Glucose Test’’). Through this integration, the task of retrieved from their records, converted to the appro-
rule authoring is insulated from the occasional, and priate forms, and passed to the guideline programs to
even day-to-day, changes that occur in the terminol- obtain results with a minimum of interaction with the
ogies used to record patient data. The MED also plays guideline logic.
a role in the end-to-end process of parsing and coding
Data Mining
test reports for evaluation by rules searching for clin-
ical conditions. In one study, by Hripcsak et al.,38 the The clinical record holds knowledge that has im-
system reliably detected the evidence of six conditions plications beyond the care of individual patients.
of interest in 200 reports at a rate that was indistin- By studying patterns and trends through a process
guishable from expert human reviewers. known as data mining, it is possible to generate new
medical knowledge from patient data. The MED has
A long-held model for applying knowledge to patient supported such efforts directly though its coding
care has been the expert system, in which expert of the patient record. For example, Wilcox and
knowledge is encoded in a system and brought to Hripcsak44 have used the MED, together with natural
294 CIMINO, From Data to Knowledge

F i g u r e 5 Integrations of
DXplain with a clinical in-
formation system. Top, The
numeric results from a
chemistry panel have been
converted to specific clinical
findings to be passed to
DXplain. Bottom, The differ-
ential diagnosis obtained
from DXplain.

language processing, to identify patient records of in- cilitate the browsing and visualization tasks needed
terest for clinical studies. These researchers have also by terminology editors.47 These tools have been used
used knowledge in the MED to construct tools that successfully to help correct errors and inconsistencies
use terminological knowledge to help would-be data in the MED and to improve its comprehensibility.48
miners understand what data are in the medical rec-
ord, how they are coded, and how best to retrieve Discussion
them.45
Knowledge representation in medical informatics has
Terminology Maintenance and Use a rich history. While much of the previous work has
been focused on how to organize symbols into knowl-
The knowledge in the MED was originally included
edge, the representation of the symbols themselves
to support intelligent terminology maintenance tools.
has turned out to be as important, if not more impor-
This knowledge has, indeed, been used for this pur-
tant, for supporting the use of knowledge in practical
pose. Terminologies from disparate laboratory sys-
systems. One can theorize that the lackluster adoption
tems at Presbyterian Hospital were successfully
of artificial intelligence systems in health care may be
merged and a tool was created to support automated
due in part to failure to ascribe proper importance to
update of the pharmacy terminology in the MED. This
‘‘getting things by their right name.’’ In any case,
tool also proved useful for detecting discrepancies in
many terminology developers today are committing
the pharmacy’s terminology, particularly with regard
extensive resources to the task of knowledge repre-
to missing allergy information.46
sentation because they believe that this approach will
More recently, the MED has supported the integration serve them well in managing their products and serve
of information from disparate systems as Presbyterian their clients well in using their products. Time will tell
Hospital merged with New York Hospital (unpub- whether the extra effort is worthwhile. The recently
lished data). Other tools have been developed to fa- announced merger of the Read Codes and SNOMED49
Journal of the American Medical Informatics Association Volume 7 Number 3 May / Jun 2000 295

will be particularly interesting to watch, since each uable information about the human condition that can
includes definitional knowledge that is (theoretically) inform clinical research. However, to mine the gems
interchangeable and may support the merging pro- from these data, we need to know where to look and
cess. how to recognize what we find. At least in the MED’s
case, knowledge-based approaches are helping sup-
The MED at Columbia University and the New York
port both these tasks.44,45
Presbyterian Hospital is but one example of a knowl-
edge-based terminology, and a rather modest one in Finally, the development of complex medical appli-
comparison with current efforts elsewhere. However, cations to support patient care demands its own type
it does serve as a proof-of-concept for the general ap- of knowledge about how all the pieces fit together. In
proach, and we have had a decade of experience in our case, this includes the task of maintaining the
building and using it. From that experience, we can MED knowledge itself. The incorporation of such
offer anecdotal evidence that the effort to include knowledge into the MED has implications beyond its
knowledge in a terminology is, indeed, worth it. The simple symmetry; it facilitates the use of expert sys-
knowledge in the terminology lets us take coded pa- tems to audit the knowledge and applications to ver-
tient data and arrive at knew knowledge in several ify that they will function as intended. The example
ways. of discovering missing allergy information in the
pharmacy system is of more than theoretic interest: it
First, we can gain knowledge about the patient. Al- is a concrete example of how MED knowledge about
though clinicians may claim that they need to know itself can have a potentially life-saving impact on pa-
all the data about a patient to make appropriate de- tient care.
cisions, human memory is simply no match for the
amount of information modern medicine is capable of Although not originally intended as a data dictionary,
generating about a complex patient. By using knowl- information about the clinical repository’s tables and
edge about the meaning of the data, we can retrieve, columns, and their interdependencies, has been added
filter, and organize them in more intelligent ways, to the MED. This knowledge has the potential to sup-
which are appropriate to the task at hand and reduce port database administrators and system developers
cognitive overload.33 in their understanding of how coded data relate to the
database structure (S. B. Johnson, personal commu-
Second, patient data are potentially useful for point- nication, 1999). The advantages of having the data-
ing us to relevant information resources; they are base model represented as a collection of concepts,
more likely to be useful if they can be translated or integrated with the concepts stored in the database,
mapped to a form that can be used to search a re- seem likely. For example, subsets of the MED can be
source. For example, a serum sodium test result of 120 reused in different parts of the database. Also, if the
cannot be used to retrieve useful MEDLINE citations by data model is changed, the impact on the terminology
searching PubMed for ‘‘120’’ or even by searching for should be apparent, and vice versa. However, it is too
‘‘serum sodium test.’’ Knowing that the test result is early to tell how this form of knowledge will prove
related to a term that is recognized by PubMed, such most useful.
as ‘‘hyponatremia,’’ provides the necessary bridge be-
tween patient data and the knowledge available in the Terminology requirements, as stated by researchers in
medical literature. terminological work, were recently collected and sum-
marized as set of ‘‘desiderata.’’ 52 Two short years ago,
Third, we finally have an opportunity to bring expert I was unable to predict ‘‘whether the semantic, defi-
systems to bear directly on the task of patient care. nitional information provided by [terminology] de-
These systems are typically constructed using terms velopers will be minimal, complete, or somewhere in
that are appropriate for medical decisions but not between.’’ Cautious optimism now suggests that cur-
equivalent to those appearing in the medical record.50 rent efforts are moving toward the ‘‘complete’’ end of
As a result, using expert systems requires human the spectrum. Getting there will require change, com-
translation and transfer of the information from the promise, and overcoming technical, epistemological,
medical record to the system.51 The ability to translate and political hurdles. As we move forward, we will
terms as originally envisioned by the UMLS devel- do well to recall the namesake for the terminology
opers,26 coupled with the ease of integrating applica- desiderata:
tions on the Web, offers exciting potential for expert
systems to find practical use in everyday patient Go placidly amid the noise and haste, and remember
care. what peace there may be in silence. As far as possible,
without surrender, be on good terms with all persons.
Fourth, the medical records of patients contain inval- —Desiderata, MAX EHRMANN, 1927
296 CIMINO, From Data to Knowledge

The author thanks all his colleagues at the Columbia University 15. Bakken S, Cashen MS, Mendonça EA, O’Brien A, Zieniewicz
Department of Medical Informatics for working with him to J. Representing nursing activities within a concept-oriented
expand and exploit the knowledge in the MED in innovative terminological system: evaluation of a type definition. J Am
and exciting ways, including Paul Clayton, George Hripcsak, Med Inform Assoc. 2000;7:81–90.
Steve Johnson, Soumitra Sengupta, Carol Friedman, Bob Sideli, 16. Rector AL, Nowlan WA, Glowinski A. Goals for concept
Justin Starren, Randy Barrows, Bruce Forman, Barry Allen, Rob- representation in the GALEN project. Proc Annu Symp
ert Jenders, Gai Elhanan, Qing Zeng, Adam Wilcox, and Eneida Comput Appl Med Care. 1994:414–8.
Mendonça. He also thanks Sue Bakken for the inspiration to 17. Brown PJB, O’Neil M, Price C. Semantic definition of dis-
write on this topic, Nancy Lorenzi for the opportunity to pres- orders in version 3 of the Read Codes. Methods Inf Med.
ent it at the 1999 AMIA Fall Symposium, and Andria Brummitt 1998;37(4–5):415–9.
for editorial support. 18. Hardiker NR, Rector AL. Modeling nursing terminology us-
ing the GRAIL representation language. J Am Med Inform
Assoc. 1998;5(1):120–8.
References n 19. Huff SM, Rocha RA, McDonald CJ, et al. Development of
the Logical Observations Identifiers, Names, and Codes
1. Cimino JJ, Hripcsak G, Johnson SB and Clayton PD. De- (LOINC) vocabulary. J Am Med Inform Assoc. 1998;5(3):
signing an introspective, controlled medical vocabulary. 276–92.
Proc Annu Symp Comput Appl Med Care. 1989:513–8. 20. Baorto DM, Cimino JJ, Parvin CA, Kahn MG. Using Logical
2. Ledley R, Lusted L. Reasoning foundations of medical di- Observations, Identifiers, Names, and Codes (LOINC) to
agnosis. Science. 1959;130:9–21. exchange laboratory data among three academic hospitals.
3. Shortliffe EH, Axline SG, Buchanan BG, Merigan TC, Cohen Proc AMIA Annu Fall Symp. 1997:96–100.
SN. An artificial intelligence program to advise physicians 21. Lau LM, Lam SH. Comparing the mapping of laboratory
regarding antimicrobial therapy. Comput Biomed Res. 1973; results to LOINC among different healthcare institutions us-
6(6):544–60. ing a common data dictionary. Proc AMIA Symp. 1999:1105.
4. Starren J, Xie Q. Comparison of three knowledge represen- 22. Rocha RA, Huff SM, Haug PJ, Warner HR. Designing a con-
tation formalisms for encoding the NCEP cholesterol guide- trolled medical vocabulary server: the VOSER project. Com-
lines. Proc Annu Symp Comput Appl Med Care. 1994: put Biomed Res. 1994;27(6):472–507.
792–6. 23. Rector AL, Solomon WD, Nowland WA, et al. A terminol-
5. van der Lei J, Musen MA. Separation of critiquing knowl- ogy server for medical language and medical information
edge from medical knowledge: implications for the Arden systems. Methods Inf Med. 1995;34(1–2):147–57.
Syntax. In Timmers T, Blum BI (eds). Proceedings of the 24. Chute CG, Elkin PL, Sherertz DD, Tuttle MS. Desiderata for
International Medical Informatics Association Working a clinical terminology server. Proc AMIA Symp. 1999:42–6.
Conference on Software Engineering in Medical Informat- 25. Mays E, Weida R, Laker M, et al. Scalable and expressive
ics. Amsterdam, The Netherlands: North-Holland Press, medical terminologies. Proc AMIA Annu Fall Symp. 1996:
1990:499–509. 259–63.
6. Campbell KE, Das AK, Musen MA. A logical foundation for 26. Lindberg DA, Humphreys BL, McCray AT. The Unified
representation of clinical data. J Am Med Inform Assoc. Medical Language System. Methods Inf Med. 1993;32(4):
1994;1:218–32. 281–91.
7. Chute CG. The Copernican era of healthcare terminology: 27. Humphreys BL, Lindberg DA, Schoolman HM, Barnett GO.
a re-centering of health information systems. Proc AMIA The Unified Medical Language System: an informatics re-
Symp. 1998:68–73. search collaboration. J Am Med Inform Assoc. 1998;5(1):
8. Cimino JJ, Barnett GO. Automated translation between 1–11.
medical terminologies using semantic definitions. Proceed- 28. Cimino JJ, Clayton PD, Hripcsak G, Johnson SB. Knowl-
ings of the American Association for Medical Systems and edge-based approaches to the maintenance of a large con-
Informatics Congress. 1989:113–7. Also in: MD Comput. trolled medical terminology. J Am Med Inform Assoc. 1994;
1990;7(2):104–9. 1(1):35–50.
9. Masarie FE Jr, Miller RA, Bouhaddou O, Giuse NB, Warner 29. Johnson S, Friedman C, Cimino JJ, Clark T, Hripcsak G,
HR. An interlingua for electronic interchange of medical in- Clayton PD. Conceptual data model for a central patient
formation: using frames to map between clinical vocabu- database. Proc Annu Symp Comput Appl Med Care. 1991:
laries. Comput Biomed Res. 1991;24(4):379–400. 381–385.
10. Bernauer J. A controlled vocabulary framework for report 30. Barrows RC, Allen BA, Sherman E, Smith K. A Decision-
generation in bone scintigraphy. Proc Annu Symp Comput supported outpatient practice system. Proc AMIA Annu Fall
Appl Med Care. 1990:195–9. Symp. 1996:792–6.
11. Bernauer J. Conceptual graphs as an operational model for 31. Hripcsak G, Cimino JJ, Sengupta S. WebCIS: large-scale de-
descriptive findings. Proc Annu Symp Comput Med Care. ployment of a Web-based clinical information system. Proc
1991:214–8. AMIA Symp. 1999:804–8.
12. Campbell KE, Musen MA. Representation of clinical data 32. Elhanan G, Cimino JJ. Controlled vocabulary and design of
using SNOMED III and conceptual graphs. Proc Annu Symp laboratory results displays. Proc AMIA Annu Fall Symp.
Comput Appl Med Care. 1993:354–8. 1997:719–23.
13. Campbell KE, Cohn SP, Chute CG, Rennels G, Shortliffe EH. 33. Zeng Q, Cimino JJ. Evaluation of a system to identify rele-
Gálapagos: computer-based support for evolution of a con- vant patient information and its impact on clinical infor-
vergent medical terminology. Proc AMIA Annu Fall Symp. mation retrieval. Proc AMIA Symp. 1999:642–8.
1996:269–73. 34. Cimino JJ, Johnson SB, Aguirre A, Roderer N, Clayton PD.
14. Spackman KA, Campbell KE, Côté RA. SNOMED RT: a ref- The MEDLINE button. Proc 16th Annu Symp Comput Appl
erence terminology for health care. Proc AMIA Annu Fall Med Care. 1992:81–5.
Symp. 1997:640–4. 35. Cimino JJ, Elhanan G, Zeng Q. Supporting Infobuttons with
Journal of the American Medical Informatics Association Volume 7 Number 3 May / Jun 2000 297

terminological knowledge. Proc AMIA Annu Fall Symp. 44. Wilcox AB, Hripcsak G. Classification algorithms applied to
1997:528–32. narrative reports. Proc AMIA Symp. 1998:455–9.
36. Zeng Q, Cimino JJ. Linking a clinical system to heteroge- 45. Wilcox AB, Hripcsak G, Chen C. Creating an environment
neous information resources. Proc AMIA Annu Fall Symp. for linking knowledge-based systems to a clinical database:
1997:553–7. a suite of tools. Proc AMIA Annu Fall Symp. 1997:303–7.
37. Hripcsak G, Clayton PD, Jenders RA, Cimino JJ, Johnson 46. Cimino JJ, Johnson SB, Hripcsak G, Hill CL, Clayton PD.
SB. Design of a clinical event monitor. Comput Biomed Res. Managing vocabulary for a centralized clinical system.
1996;29(3):194–221. Medinfo. 1995:117–20.
38. Hripcsak G, Friedman C, Alderson PO, DuMouchel W, 47. Gu H, Halper M, Geller J, Cimino JJ, Perl Y. Utilizing OODB
Johnson SB, Clayton PD. Unlocking clinical data from nar- schema modeling for vocabulary management. Proc AMIA
rative reports: a study of natural language processing. Ann Annu Fall Symp. 1996:274–8.
Intern Med. 1955;122(9):681–8. 48. Gu H, Halper M, Geller J, Perl Y. Benefits of an object-ori-
39. Elhanan G, Socratous DA, Cimino JJ. Integrating DXplain ented database representation for controlled medical ter-
into a clinical information system using the World Wide minologies. J Am Med Inform Assoc. 1999;6:283–303.
Web. Proc AMIA Annu Fall Symp. 1996:348–52. 49. College of American Pathologists. SNOMEDw RT and READ
40. Barnett GO, Cimino JJ, Hupp JA, Hoffer EP. DXplain: an Codes to Be Combined into an International Terminol-
evolving diagnostic decision-support system. J Am Med As- ogy of Health [press release]. May 19, 1999. CAP Web site.
soc. 1987;258(1):67–74. Available at: http://www.cap.org/html/public/snomedo
41. Cimino JJ, Socratous SA, Clayton PD. Automated guidelines intl.html. Accessed Mar 15, 2000.
implemented via the World Wide Web [poster]. Proc Annu 50. Wong ET, Pryor TA, Huff SM, Hau PJ, Warner HR. Inter-
Symp Comput Appl Med Care. 1995:941. facing a stand-alone diagnostic expert system with a hos-
42. Chen Y, Wang SS, Cimino JJ. Linking guidelines for mam- pital information system. Comput Biomed Res. 1994;27:
mography to an electronic medical record for use by pa- 116–29.
tients. Proc AMIA Symp. 1999:1041. 51. Miller RA, McNeil MA, Challinor SM, Masarie FE, Myers
43. Cimino JJ, Sengupta A, Clayton PD, Patel VL, Kushniruk JD. The Quick Medical Reference Project: status report. West
AW, Huang X. Architecture for a Web-based clinical infor- J Med. 1986;145:816–22.
mation system that keeps the design open and the access 52. Cimino JJ. Desiderata for controlled medical vocabularies in
closed. Proc AMIA Symp. 1998:121–5. the 21st century. Methods Inf Med. 1998;37(4–5):394–403.