The HistCorp Collection of Historical Corpora and Resources

2012

Language technology tools can be very useful for making information concealed in historical documents more easily accessible to historians, linguists and other researchers in humanities. For many languages, there is however a lack of linguistically annotated historical data that could be used for training NLP tools adapted to historical text. One way of avoiding the data sparseness problem in this context is to normalise the input text to a more modern spelling, before applying NLP tools trained on contemporary corpora. In this paper, we explore the impact of a set of hand-crafted normalisation rules on Swedish texts ranging from 1527 to 1812. Normalisation accuracy as well as tagging and parsing performance are evaluated. We show that, even though the rules were generated on the basis of one 17th century text sample, the rules are applicable to all texts, regardless of time period and text genre. This clearly indicates that spelling correction is a useful strategy for applying contemporary NLP tools to historical text.

Corpus linguistics has revolutionised our way of working in historical linguistics. The painstaking job of collecting data and manually analysing them has been made less arduous with the introduction of the machine processing of corpora, which allows for quick and efficient searches. The aim of the present study is two-fold: to show how corpus linguistics has contributed to the ways in which researchers approach the study of the history of English, and to provide an overview of selected corpora available in the field. Setting aside the theoretical debate as to whether corpus linguistics should be considered merely a methodology, a branch of linguistics, or both (Taylor, 2008), it is widely acknowledged that corpus linguistics is of considerable help in any branch of linguistics, be it theoretical or applied. The use of corpora makes it possible to test hypotheses established within a specific linguistic area through the fast and reliable analysis of vast pools of material. As a result, the objective measurement of data is available to scholars, who can thus verify their hypotheses and intuitions, and can quickly amend or qualify their research claims if previous ones are seen to be falsifiable. There is, then, a continuous interaction within theory, as expressed in linguistic postulates, concepts and hypotheses, and an application and validation of these theoretical principles through the use of linguistic corpus analysis. The use of corpora is perhaps a more powerful instrument in the field of historical linguistics than in other fields, since the absence of living informants here makes judgements based on intuitions unreliable, and claims have to be empirically attested using data. This data can be extracted from systematically compiled collections of machine-readable texts, called corpora. However, in considering these undeniably advantageous working tools, some caveats should be borne in mind, as will be discussed in what follows.

2016

Historical text constitutes a rich source of information for historians and other researchers in humanities. Many texts are however not available in an electronic format, and even if they are, ther ...

ArXiv, 2013

The impact-es diachronic corpus of historical Spanish compiles over one hundred books —containing approximately 8 million words— in addition to a complementary lexicon which links more than 10 thousand lemmas with attestations of the different variants found in the documents. This textual corpus and the accompanying lexicon have been released under an open license (Creative Commons by-nc-sa) in order to permit their intensive exploitation in linguistic research. Approximately 7% of the words in the corpus (a selection aimed at enhancing the coverage of the most frequent word forms) have been annotated with their lemma, part of speech, and modern equivalent. This paper describes the annotation criteria followed and the standards, based on the Text Encoding Initiative recommendations, used to the represent the texts in digital form. As an illustration of the possible synergies between diachronic textual resources and linguistic research, we describe the application of statistical mach...

Challenging the Myth of Monolingual Corpora, 2017

Digital Scholarship in the Humanities, 2014

We describe, evaluate, and improve the automatic annotation of diachronic corpora at the levels of word-class, lemma, chunks, and dependency syntax. As corpora we use the ARCHER corpus (texts from 1,600 to 2,000) and the ZEN corpus (texts from 1,660 to 1,800). Performance on Modern English is considerably lower than on Present Day English (PDE). We present several methods that improve performance. First we use the spelling normalization tool VARD to map spelling variants to their PDE equivalent, which improves tagging. We investigate the tagging changes that are due to the normalization and observe improvements, deterioration, and missing mappings. We then implement an optimized version, using VARD rules and preprocessing steps to improve normalization. We evaluate the improvement on parsing performance, comparing original text, standard VARD, and our optimized version. Over 90% of the normalization changes lead to improved parsing, and 17.3% of all 422 manually annotated sentences get a net improved parse. As a next step, we adapt the parser's grammar, add a semantic expectation model and a model for prepositional phrases (PP)-attachment interaction to the parser. These extensions improve parser performance, marginally on PDE, more considerably on earlier texts-2-5% on PP-attachment relations (e.g. from 63.6 to 68.4% and from 70 to 72.9% on 17th century texts). Finally, we briefly outline linguistic applications and give two examples: gerundials and auxiliary verbs in the ZEN corpus, showing that despite high noise levels linguistic signals clearly emerge, opening new possibilities for large-scale research of gradient phenomena in language change.

2015

The focus of this keynote is the larger picture of how different disciplines of classics and archaeologies from analogue times, when taken onto the digital level, change. And what that change looks like, when focused on the often misunderstood role of computer linguists inside the data ecosystem of classics and archaeologies.

Literary and Linguistic Computing, 2007

Universität Duisburg-Essen) Paul Rayson (Lancaster University) 1. Introduction In this paper, we describe the approaches taken by two teams of researchers to the identification of spelling variants. Each team is working on a different language (English and German) but both are using historical texts from much the same time period (17 th -19 th century). The approaches differ in a number of other respects, for example we can draw a distinction between two types of context rules: in the German system, context rules operate at the level of individual letters and represent constraints on candidate letter replacements or n-graphs; in the English system, contextual rules operate at the level of words and provide clues to detect real-word spelling variants i.e. 'then' used instead of 'than'. However, we noticed an overlap between the types of issues that we need to address for both English and German and also a similarity between the letter replacement patterns found in the two languages.

Language Resources and Evaluation, 2013

The impact-es diachronic corpus of historical Spanish compiles over one hundred books-containing approximately 8 million words-in addition to a complementary lexicon which links more than 10 thousand lemmas with attestations of the different variants found in the documents. This textual corpus and the accompanying lexicon have been released under an open license (Creative Commons by-nc-sa) in order to permit their intensive exploitation in linguistic research. Approximately 7% of the words in the corpus (a selection aimed at enhancing the coverage of the most frequent word forms) have been annotated with their lemma, part of speech, and modern equivalent. This paper describes the annotation criteria followed and the standards, based on the Text Encoding Initiative recommendations, used to the represent the texts in digital form. As an illustration of the possible synergies between diachronic textual resources and linguistic research, we describe the application of statistical machine translation techniques to infer probabilistic context-sensitive rules for the automatic modernisation of spelling. The automatic modernisation with this type of statistical methods leads to very low character error rates when the output is compared with the supervised modern version of the text.

The aim of this paper is to offer a description of the Corpus of Historical English Texts (CHET), one of the several sub-corpora within the Coruña Corpus of English Scientific Writing (CC). The compilation principles behind it as well as the sociolinguistic variables considered in the process of text selection will be explained.