The computational analysis of documents to learn about their authorship (also known as authorship... more The computational analysis of documents to learn about their authorship (also known as authorship attribution and/or authorship profiling) is an increasingly important area of research and application of technology. This paper discusses the technology , focusing on its application to social media in a variety of disciplines. It includes a brief survey of the history as well as three tutorial case studies, and discusses several significant applications and societal benefits that authorship analysis has brought about. It further argues, though, that while the benefits of this technology have been great, it has created serious risks to society that have not been sufficiently considered, addressed, or mitigated.
Exploring Cognition: Damaged Brains and Neural Networks: Readings in Cognitive Neuropsychology and Connectionist Modelling, 2000
The conventional interpretation of double dissociations is that they are almost irrefutable evide... more The conventional interpretation of double dissociations is that they are almost irrefutable evidence of distinctions in both function and type of mental processes, or of separation of cognition into modules. We present a connectionist model that demonstrates apparent double dissociations within a single-route, single-mechanism network and argue that these apparent dissociations are simply the expected tails of a standard bell curve describing network performance. We conclude that within a connectionist model, the appearance of double dissociations may not be evidence for functional or mechanistic separation, and that similar caveats apply to the interpretation of double dissociations in human cognitive behaviour.
The author identification task at PAN-2014 focuses on author verification. Similar to PAN-2013 we... more The author identification task at PAN-2014 focuses on author verification. Similar to PAN-2013 we are given a set of documents by the same author along with exactly one document of questioned authorship, and the task is to determine whether the known and the questioned documents are by the same author or not. In comparison to PAN-2013, a significantly larger corpus was built comprising hundreds of documents in four natural languages (Dutch, English, Greek, and Spanish) and four genres (essays, reviews, novels, opinion articles). In addition, more suitable performance measures are used focusing on the accuracy and the confidence of the predictions as well as the ability of the submitted methods to leave some problems unanswered in case there is great uncertainty. To this end, we adopt the c@1 measure, originally proposed for the question answering task. We received 13 software submissions that were evaluated in the TIRA framework. Analytical evaluation results are presented where one language-independent approach serves as a challenging baseline. Moreover, we continue the successful practice of the PAN labs to examine meta-models based on the combination of all submitted systems. Last but not least, we provide statistical significance tests to demonstrate the important differences between the submitted approaches.
Abstract A possible problem with many of the large-corpus tech-niques used for document categoriz... more Abstract A possible problem with many of the large-corpus tech-niques used for document categorization or similarity judgements is the very fact that they require large cor-pora for reliability. A powerful test against the distilled wisdom of hundreds of millions or billions of words may be of ...
Proceeding of Fourth International Conference on Spoken Language Processing Icslp 96, Oct 3, 1996
Like many cryptosystems, PGPfone[13] requires a method of reliably exchanging binary data over no... more Like many cryptosystems, PGPfone[13] requires a method of reliably exchanging binary data over noisy phone lines. This paper describes a method of encoding binary data into a \radio alphabet," using a feature-based distance metric to measure phonetic confusibility, then using this metric in a GA to select appropriate words from a larger list of candidate words. This work indicates several larger issues that should be addressed in any ( h uman) language engineering project.
Proceedings of the Joint Conferences on New Methods in Language Processing and Computational Natural Language Learning, Jan 11, 1998
The idea of '~familial relationships" among languages is well-established and accepted, although ... more The idea of '~familial relationships" among languages is well-established and accepted, although some controversies persist in a few specific instances. By painstakingly recording and identifying regularities and similarities and comparing these to the historical record, linguists have been able to produce a general "family tree" incorporating most natural languages. We suggest here that much of these trees can be automatically determined by a complementary technique of distributional analysis. Recent work by and (Juola, 1997) suggests that Kullback-Leibler divergence (or cross-entropy) can be meaningfully measured from small samples, in some cases as small as only 20 or so words. Using these techniques, we define and measure a distance function between translations of a small corpus (c. 70 words/sample) covering much of the accepted Indo-European family, and reconstruct a relationship tree by hierarchical duster analysis. The resulting tree shows remarkable similarity to the accepted Indo-European family; this we read as evidence both for the immense power of this measurement technique and for the validity of this kind of mechanical similarity judgement in the identification of typological relationships. Furthermore, this technique is in theory sensitive to different sorts of relationships than more common word-list based methods and may help ilium;hate these from a different direction.
We explore mutual information (MI) as a means of characterizing linguistic statistical structure.... more We explore mutual information (MI) as a means of characterizing linguistic statistical structure. The MI between two linguistic tokens x and y is the degree to which seeing x helps us anticipate the occurrence of y. We computed MI between words in 595 samples of written text in 25 languages. Our analyses indicate that MI dependencies do not extend beyond a range of five words. Moreover, the similarity between MI profiles of different languages was used to cluster the languages. These results are discussed in terms of a putative link between short-term memory and linguistic structure and the further utility of MI in terms of characterizing the latter.
Computers and the Humanities Official Journal of the Association For Computers and the Humanities, 2003
This paper presents a numeric and information theoretic model for the measuring of language chang... more This paper presents a numeric and information theoretic model for the measuring of language change, without specifying the particular type of change. It is shown that this measurement is intuitively plausible and that meaningful measurements can be made from as few as 1000 characters. This measurement technique is extended to the task of determining the "rate" of language change based on an examination of brief excerpts from the National Geographic Magazine and determining both their linguistic distance from one another as well as the number of years of temporal separation. A statistical analysis of these results shows, first, that language change can be measured, and second, that the rate of language change has not been uniform, and that in particular, the period 1939-1948 had particularly slow change, while 1949-1958 and 1959-1968 had particularly rapid changes.
Proceedings of the Workshop on Computational Approaches to Deception Detection, Apr 23, 2012
... Put more strongly, when “Chris”(or any of the other test subjects) attempted to write in the ... more ... Put more strongly, when “Chris”(or any of the other test subjects) attempted to write in the style of Cormac McCarthy, the result was actually ... Zimmermann (nd) has suggested that “it would be nice if everyone routinely used encryp-tion for all their E-mail, innocent or not, so that ...
This paper describes an attempt to combine the advantages of both example-based translation and s... more This paper describes an attempt to combine the advantages of both example-based translation and stochastic translation methods in an attempt to develop a method for inferring symbolic transfer functions from a bilingual corpus. By formalizing the translation process and applying standard optimization techniques, a system can be developed that will identify grammatical categories and produce coherent transfer functions between languages. The validity of this approach is demonstrated in a prototype system that can learn transfer functions between English, French, and Urdu.
The recent growth in digital scholarship has made literally millions of books available to reader... more The recent growth in digital scholarship has made literally millions of books available to readers. But the implications of this, paradoxically, are that reading becomes more difficult. No human can possibly read and understand a million books. This is particularly problematic in literary scholarship, where "reading" a text requires much more than simple content extraction, but may require identifying and explaining patterns of thought and expression across many different works.
Abstract Traditionally, regularity in a data set is assessed by t-ting a model to the data and ex... more Abstract Traditionally, regularity in a data set is assessed by t-ting a model to the data and examining the extent to which the variance accounted for by the model is large compared to the overall variance in the system. Such approaches, however, do not address the complementary ...
Journal of the Chicago Colloquium on Digital Humanities and Computer Science, Jul 19, 2009
Unfortunately, the author was anonymous. Finding out who this author was would go a long way to a... more Unfortunately, the author was anonymous. Finding out who this author was would go a long way to assessing his or her credibility. The task of determining from a document's text who wrote it has a long history but no clearly-defined best solution. We present a software system ( ...
The computational analysis of documents to learn about their authorship (also known as authorship... more The computational analysis of documents to learn about their authorship (also known as authorship attribution and/or authorship profiling) is an increasingly important area of research and application of technology. This paper discusses the technology , focusing on its application to social media in a variety of disciplines. It includes a brief survey of the history as well as three tutorial case studies, and discusses several significant applications and societal benefits that authorship analysis has brought about. It further argues, though, that while the benefits of this technology have been great, it has created serious risks to society that have not been sufficiently considered, addressed, or mitigated.
Exploring Cognition: Damaged Brains and Neural Networks: Readings in Cognitive Neuropsychology and Connectionist Modelling, 2000
The conventional interpretation of double dissociations is that they are almost irrefutable evide... more The conventional interpretation of double dissociations is that they are almost irrefutable evidence of distinctions in both function and type of mental processes, or of separation of cognition into modules. We present a connectionist model that demonstrates apparent double dissociations within a single-route, single-mechanism network and argue that these apparent dissociations are simply the expected tails of a standard bell curve describing network performance. We conclude that within a connectionist model, the appearance of double dissociations may not be evidence for functional or mechanistic separation, and that similar caveats apply to the interpretation of double dissociations in human cognitive behaviour.
The author identification task at PAN-2014 focuses on author verification. Similar to PAN-2013 we... more The author identification task at PAN-2014 focuses on author verification. Similar to PAN-2013 we are given a set of documents by the same author along with exactly one document of questioned authorship, and the task is to determine whether the known and the questioned documents are by the same author or not. In comparison to PAN-2013, a significantly larger corpus was built comprising hundreds of documents in four natural languages (Dutch, English, Greek, and Spanish) and four genres (essays, reviews, novels, opinion articles). In addition, more suitable performance measures are used focusing on the accuracy and the confidence of the predictions as well as the ability of the submitted methods to leave some problems unanswered in case there is great uncertainty. To this end, we adopt the c@1 measure, originally proposed for the question answering task. We received 13 software submissions that were evaluated in the TIRA framework. Analytical evaluation results are presented where one language-independent approach serves as a challenging baseline. Moreover, we continue the successful practice of the PAN labs to examine meta-models based on the combination of all submitted systems. Last but not least, we provide statistical significance tests to demonstrate the important differences between the submitted approaches.
Abstract A possible problem with many of the large-corpus tech-niques used for document categoriz... more Abstract A possible problem with many of the large-corpus tech-niques used for document categorization or similarity judgements is the very fact that they require large cor-pora for reliability. A powerful test against the distilled wisdom of hundreds of millions or billions of words may be of ...
Proceeding of Fourth International Conference on Spoken Language Processing Icslp 96, Oct 3, 1996
Like many cryptosystems, PGPfone[13] requires a method of reliably exchanging binary data over no... more Like many cryptosystems, PGPfone[13] requires a method of reliably exchanging binary data over noisy phone lines. This paper describes a method of encoding binary data into a \radio alphabet," using a feature-based distance metric to measure phonetic confusibility, then using this metric in a GA to select appropriate words from a larger list of candidate words. This work indicates several larger issues that should be addressed in any ( h uman) language engineering project.
Proceedings of the Joint Conferences on New Methods in Language Processing and Computational Natural Language Learning, Jan 11, 1998
The idea of '~familial relationships" among languages is well-established and accepted, although ... more The idea of '~familial relationships" among languages is well-established and accepted, although some controversies persist in a few specific instances. By painstakingly recording and identifying regularities and similarities and comparing these to the historical record, linguists have been able to produce a general "family tree" incorporating most natural languages. We suggest here that much of these trees can be automatically determined by a complementary technique of distributional analysis. Recent work by and (Juola, 1997) suggests that Kullback-Leibler divergence (or cross-entropy) can be meaningfully measured from small samples, in some cases as small as only 20 or so words. Using these techniques, we define and measure a distance function between translations of a small corpus (c. 70 words/sample) covering much of the accepted Indo-European family, and reconstruct a relationship tree by hierarchical duster analysis. The resulting tree shows remarkable similarity to the accepted Indo-European family; this we read as evidence both for the immense power of this measurement technique and for the validity of this kind of mechanical similarity judgement in the identification of typological relationships. Furthermore, this technique is in theory sensitive to different sorts of relationships than more common word-list based methods and may help ilium;hate these from a different direction.
We explore mutual information (MI) as a means of characterizing linguistic statistical structure.... more We explore mutual information (MI) as a means of characterizing linguistic statistical structure. The MI between two linguistic tokens x and y is the degree to which seeing x helps us anticipate the occurrence of y. We computed MI between words in 595 samples of written text in 25 languages. Our analyses indicate that MI dependencies do not extend beyond a range of five words. Moreover, the similarity between MI profiles of different languages was used to cluster the languages. These results are discussed in terms of a putative link between short-term memory and linguistic structure and the further utility of MI in terms of characterizing the latter.
Computers and the Humanities Official Journal of the Association For Computers and the Humanities, 2003
This paper presents a numeric and information theoretic model for the measuring of language chang... more This paper presents a numeric and information theoretic model for the measuring of language change, without specifying the particular type of change. It is shown that this measurement is intuitively plausible and that meaningful measurements can be made from as few as 1000 characters. This measurement technique is extended to the task of determining the "rate" of language change based on an examination of brief excerpts from the National Geographic Magazine and determining both their linguistic distance from one another as well as the number of years of temporal separation. A statistical analysis of these results shows, first, that language change can be measured, and second, that the rate of language change has not been uniform, and that in particular, the period 1939-1948 had particularly slow change, while 1949-1958 and 1959-1968 had particularly rapid changes.
Proceedings of the Workshop on Computational Approaches to Deception Detection, Apr 23, 2012
... Put more strongly, when “Chris”(or any of the other test subjects) attempted to write in the ... more ... Put more strongly, when “Chris”(or any of the other test subjects) attempted to write in the style of Cormac McCarthy, the result was actually ... Zimmermann (nd) has suggested that “it would be nice if everyone routinely used encryp-tion for all their E-mail, innocent or not, so that ...
This paper describes an attempt to combine the advantages of both example-based translation and s... more This paper describes an attempt to combine the advantages of both example-based translation and stochastic translation methods in an attempt to develop a method for inferring symbolic transfer functions from a bilingual corpus. By formalizing the translation process and applying standard optimization techniques, a system can be developed that will identify grammatical categories and produce coherent transfer functions between languages. The validity of this approach is demonstrated in a prototype system that can learn transfer functions between English, French, and Urdu.
The recent growth in digital scholarship has made literally millions of books available to reader... more The recent growth in digital scholarship has made literally millions of books available to readers. But the implications of this, paradoxically, are that reading becomes more difficult. No human can possibly read and understand a million books. This is particularly problematic in literary scholarship, where "reading" a text requires much more than simple content extraction, but may require identifying and explaining patterns of thought and expression across many different works.
Abstract Traditionally, regularity in a data set is assessed by t-ting a model to the data and ex... more Abstract Traditionally, regularity in a data set is assessed by t-ting a model to the data and examining the extent to which the variance accounted for by the model is large compared to the overall variance in the system. Such approaches, however, do not address the complementary ...
Journal of the Chicago Colloquium on Digital Humanities and Computer Science, Jul 19, 2009
Unfortunately, the author was anonymous. Finding out who this author was would go a long way to a... more Unfortunately, the author was anonymous. Finding out who this author was would go a long way to assessing his or her credibility. The task of determining from a document's text who wrote it has a long history but no clearly-defined best solution. We present a software system ( ...
Active authentication is the process of continuously verifying a user based on his/her ongoing in... more Active authentication is the process of continuously verifying a user based on his/her ongoing interaction with the computer. Forensic stylometry is the study of linguistic style, applied to author (user) identification. We evaluate the Active Linguistic Authentication Dataset [Juola et al., 2013], collected from users working individually in an office environment for a period of one week. We consider a battery of stylometric modalities, as a representative collection of high-level behavioral biometrics. As opposed to the initial evaluation presented on this dataset before on 14 users, we consider the fully collected dataset, which consists of data by 67 users. An additional significant difference is in the type of evaluation:
Uploads
Books by Patrick Juola
Papers by Patrick Juola