This paper introduces a semi-automatic text tagging interface to help historians efficiently garn... more This paper introduces a semi-automatic text tagging interface to help historians efficiently garner posting records from the Chinese Local Gazetteers (difangzhi 地方志) with the format of “who, when, which posting”. By turning texts into tabular data forms, this interface aims to lay the basis for analyzing Chinese local gazetteers on a large scale. Although local gazetteers from various locations all follow a general pattern when recording posting data, they still differ in details due to their substantial amount. Therefore, it is unfeasible to ask programmers to extract the posting data using a one-size-fits-all computer program. This tagging interface, on the other hand, provides a simple user interface with built-in patterns to extract the subjects’ names, posting titles, dynasties, posting times, basic addresses and entry methods. This allows users to tag most of data in the text quickly, which can then be proofread by them themselves in order to check the tagging result and to correct mistakes. The interface also enables users to adjust the extraction patterns according to each text in order to accurately extract posting data from local gazetteers with distinct patterns.
This paper introduces a digital humanities project at the Max Planck Institute for the History of... more This paper introduces a digital humanities project at the Max Planck Institute for the History of Science (Max-‐Planck-‐Institut für Wissenschaftsgeschichte, MPIWG) that aims to unlock the treasure chest of local knowledge written in the genre of Chinese local gazetteers for computer assisted analyses. In the past two decades, a great amount of historical documents have been digitized and put on the Web to enable easy access for scholars around the globe. In parallel, the amount of searchable full-‐text versions of historical texts has increased, which opens the possibility of text mining the contents for large scaled analyses. Many works in this direction have been proposed and got recognition, while they are also criticized for drawing conclusions from seemingly imprecise results due to the restriction that their algorithms have no knowledge about the meanings of the different pieces in a text (Jockers, 2013; Google Ngram Viewer; Chen et al., 2007). An alternative approach is thus to first " teach " computers what each pieces of a text means before asking computers to run automatic analyses. Such " teaching " is done by tagging, or called markup. Many digital humanities projects have been using TEI, a standard for text encoding based on XML, to tag their research materials (TEI; Flanders). In the Local Gazetteers Project, since the genre organizes knowledge in a very structural way, we are also using tagging to teach computers the meanings of texts in order to turn them into data tables to enable computer assisted analyses including GIS mapping. However, since the amount of texts is huge, we also propose a research data repository for scholars to collaborate in this project and to aggregate their results. What are the Chinese Local Gazetteers? The Chinese local gazetteers is a genre of texts that has been produced in China consistently from the 10 th century on to even today. Most of them are compiled by local officials as a major means to collect and aggregate historical, social, and geographical knowledge of an administrative region for governing purposes. There are at least 8,000 titles of pre-‐1949 local gazetteers still extant today. They cover almost every well-‐populated region in historical China. Despite being compiled by different officials for different regions, the local gazetteers have developed a pretty consistent structure of " describing " local knowledge. Most gazetteers contain the following chapters: history,
Digital humanities (DH) is a burgeoning field of research in sinology and Asian studies more broa... more Digital humanities (DH) is a burgeoning field of research in sinology and Asian studies more broadly, and its diversity and maturity necessitate a digital research infrastructure fit for DHfocused scholars' specific needs. In particular, the DH landscape evolved in a way that encourages fragmentation of both sources and tools, and these compartmentalized resources centered around disciplines and texts. RISE (formerly known as "Asia Network") is our solution to address this fragmentation across disciplines. It is a pioneering approach for resource dissemination and emerging data analytics (such as text mining and other fair-use, consumptive research techniques) in the humanities. It is a language-agnostic software that facilitates the secure linkage between third-party research tools to different third-party textual collections (both licensed and open-access ones) via application programming interfaces (APIs). Put more simply, it reduces the distances among DH resources not by duplicating them in a central repository, but by linking them together via flexible APIs. It revolutionizes how scholars can work with textual sources by promoting a flexible, networked approach to digital infrastructure development. Crucially, RISE is a loosely-coupled software with flexible topologies; it can enable both federated or centralized linkages, and it can even "disappear" as long as its API and metadata standards remain in place to facilitate communications among distributed databases and tools in the back-end. Thus, unlike largescale infrastructural projects, RISE actively lowers the profile of centralized infrastructure and instead promotes existing tools and resources by enabling their interoperability in a flexible and distributed manner. As a result, it allows scholars to fully leverage the potential of material digitization and digital research tools without recreating silos of resources in the digital realm. We believe that RISE, coupled with developing novel licensing models suited for digital research methods (including consumptive research like text mining), would significantly improve the infrastructure behind DH scholarly research in sinology and beyond.
This article gives an overview of the Local Gazetteers Research Tools (LoGaRT), including its dev... more This article gives an overview of the Local Gazetteers Research Tools (LoGaRT), including its development, technical features, methodology, and examples of research applications by members of the Tu 圖 working group. The use of LoGaRT is illustrated with four brief introductions to projects that draw on visual materials from the local gazetteers, including ritual-related illustrations, city layout maps, and maps with western cartographic features. See the websites for more detailed information on LoGaRT and other research projects using it.1
Designing a protocol for the interoperability of digital textual resources—or, more simply, a “II... more Designing a protocol for the interoperability of digital textual resources—or, more simply, a “IIIF for texts”—remains a challenge, as such a protocol must cater to their vastly heterogenous formats, structures, languages, text encodings and metadata. There have been many attempts to propose a standard for textual resource interoperability, from the ubiquitous Text Encoding Initiative (TEI) format to more recent proposals like the Distributed Text Services (DTS) protocol. In this paper, we critically survey these attempts and introduce our proposal called SHINE, which aims to escape from TEI’s legacy and prioritize instead the ease for software developers to representation and exchange textual resources and their associated metadata. We do so by combining a hierarchical model of textual structure with a flexible metadata scheme in SHINE, and we continue to define and develop it based on user-centered and iterative design principles. Therefore, we argue that SHINE is a protocol for t...
Digital humanities (DH) is a burgeoning field of research in Sinology and Asian studies more broa... more Digital humanities (DH) is a burgeoning field of research in Sinology and Asian studies more broadly, and its diversity and maturity necessitate a cyberinfrastructure fit for DH-focused Sinologists’ specific needs. “Asia Network” is our solution. It is a pioneering approach for resource dissemination and emerging data analytics (such as text mining and other fair-use, consumptive research techniques) in the humanities. It is a language-agnostic software that facilitates the secure linkage between third-party research tools to different third-party textual collections (both licensed and open-access ones) via application programming interfaces (APIs). It revolutionizes how scholars can work with textual sources by promoting a flexible, networked approach to e-infrastructure development. Crucially, Asia Network is a loosely-coupled software with flexible topologies; it can enable both federated or centralized linkages, and it can even “disappear” as long as its API standards remain in ...
Land deeds were the only proof of ownership in pre-1900 Taiwan. They are indispensable for the st... more Land deeds were the only proof of ownership in pre-1900 Taiwan. They are indispensable for the studies of Taiwan's social, anthropological, and economic evolution. We have built a full-text digital library that contains almost 40,000 land deeds. The deeds in our collection range over 250 years and are collected from over 100 sources. The unprecedented volume and diversity of the sources provide an exciting source of primary documents for historians. But they also pose an interesting challenge: how to tell if two land deeds are related. In this article, we describe an approach to discover two important relations: successive transactions and allotment agreements involving the same property. Our method enabled us to construct 6,035 such transaction pairs. We also introduce a notion of 'land transitivity graph' to capture the transitivity embedded in these transactions. We discovered 2,436 such graphs, the largest of which includes 104 deeds. Some of these graphs involve land behavior that had never been studied before.
International Journal of Humanities and Arts Computing, 2012
The Qing Imperial Court documents are a major source of primary research material for studying th... more The Qing Imperial Court documents are a major source of primary research material for studying the Qing era China since they provide the most direct and first-hand details of how national affairs were handled. However, the way Qing archived these documents has made it cumbersome to collect documents covering the same event and rebuild their original contexts. In this paper, we describe some information technology that we have developed to discover two important and useful relations among these documents. The first is the citation relation among the Imperial Edicts and the Memorials. We discovered 6,801 pairs from the 37,831 Taiwan-related Imperial Court documents in the Taiwan History Digital Library (THDL) and produced 1,101 graphs of successive citations, which we call IE-M diagrams. The second relation is a template relation, which indicates groups of documents that were created following a specific format. Numerical data can also be tabulated from these documents and be used for...
As digital humanities research relies on the digitization of sources, many of its applications ar... more As digital humanities research relies on the digitization of sources, many of its applications are based on access to data on a huge scale that makes quantitative analyses and distant reading (or a birds-eye view) possible. Based on this assumption, we show how the genre of Chinese local gazetteers, with its volume, consistent structures, and broad geographic and temporal range, provides an ideal case to benefit from the digital approach. This paper introduces the Local Gazetteers Research Tools (LoGaRT), a suite of research tools designed for studying Chinese local gazetteers based on the philosophy that any comprehensive genre, such as Chinese local gazetteers, when accompanied with tools that aim to bring a collective lens to the genre, can greatly enrich the ways that scholars approach it and can transform the genre into a research infrastructure that enables new types of research. We report on how LoGaRT opens up new perspectives for researching Chinese history by showing case ...
Building temples in order to obtain relief from natural plagues was a common religious practice i... more Building temples in order to obtain relief from natural plagues was a common religious practice in premodern societies. In historical China, citizens built locust temples in hope of avoiding locust infestations. There were no centrally collected records in historical China of such plagues or other natural disasters. In order to discern patterns in the distribution of locust plagues over time and geographical space throughout historical China, this paper replicates the work of Chinese geographer Chen Chengsiang, who used local gazetteers as major sources for collecting such data. The results of this paper include a modern gis map of locust temples based on digital editions of local gazetteers, a gis dataset, and a procedural method for constructing gis maps on other topics mentioned in local gazetteers.
This paper introduces a semi-automatic text tagging interface to help historians efficiently garn... more This paper introduces a semi-automatic text tagging interface to help historians efficiently garner posting records from the Chinese Local Gazetteers (difangzhi 地方志) with the format of “who, when, which posting”. By turning texts into tabular data forms, this interface aims to lay the basis for analyzing Chinese local gazetteers on a large scale. Although local gazetteers from various locations all follow a general pattern when recording posting data, they still differ in details due to their substantial amount. Therefore, it is unfeasible to ask programmers to extract the posting data using a one-size-fits-all computer program. This tagging interface, on the other hand, provides a simple user interface with built-in patterns to extract the subjects’ names, posting titles, dynasties, posting times, basic addresses and entry methods. This allows users to tag most of data in the text quickly, which can then be proofread by them themselves in order to check the tagging result and to correct mistakes. The interface also enables users to adjust the extraction patterns according to each text in order to accurately extract posting data from local gazetteers with distinct patterns.
This paper introduces a digital humanities project at the Max Planck Institute for the History of... more This paper introduces a digital humanities project at the Max Planck Institute for the History of Science (Max-‐Planck-‐Institut für Wissenschaftsgeschichte, MPIWG) that aims to unlock the treasure chest of local knowledge written in the genre of Chinese local gazetteers for computer assisted analyses. In the past two decades, a great amount of historical documents have been digitized and put on the Web to enable easy access for scholars around the globe. In parallel, the amount of searchable full-‐text versions of historical texts has increased, which opens the possibility of text mining the contents for large scaled analyses. Many works in this direction have been proposed and got recognition, while they are also criticized for drawing conclusions from seemingly imprecise results due to the restriction that their algorithms have no knowledge about the meanings of the different pieces in a text (Jockers, 2013; Google Ngram Viewer; Chen et al., 2007). An alternative approach is thus to first " teach " computers what each pieces of a text means before asking computers to run automatic analyses. Such " teaching " is done by tagging, or called markup. Many digital humanities projects have been using TEI, a standard for text encoding based on XML, to tag their research materials (TEI; Flanders). In the Local Gazetteers Project, since the genre organizes knowledge in a very structural way, we are also using tagging to teach computers the meanings of texts in order to turn them into data tables to enable computer assisted analyses including GIS mapping. However, since the amount of texts is huge, we also propose a research data repository for scholars to collaborate in this project and to aggregate their results. What are the Chinese Local Gazetteers? The Chinese local gazetteers is a genre of texts that has been produced in China consistently from the 10 th century on to even today. Most of them are compiled by local officials as a major means to collect and aggregate historical, social, and geographical knowledge of an administrative region for governing purposes. There are at least 8,000 titles of pre-‐1949 local gazetteers still extant today. They cover almost every well-‐populated region in historical China. Despite being compiled by different officials for different regions, the local gazetteers have developed a pretty consistent structure of " describing " local knowledge. Most gazetteers contain the following chapters: history,
Digital humanities (DH) is a burgeoning field of research in sinology and Asian studies more broa... more Digital humanities (DH) is a burgeoning field of research in sinology and Asian studies more broadly, and its diversity and maturity necessitate a digital research infrastructure fit for DHfocused scholars' specific needs. In particular, the DH landscape evolved in a way that encourages fragmentation of both sources and tools, and these compartmentalized resources centered around disciplines and texts. RISE (formerly known as "Asia Network") is our solution to address this fragmentation across disciplines. It is a pioneering approach for resource dissemination and emerging data analytics (such as text mining and other fair-use, consumptive research techniques) in the humanities. It is a language-agnostic software that facilitates the secure linkage between third-party research tools to different third-party textual collections (both licensed and open-access ones) via application programming interfaces (APIs). Put more simply, it reduces the distances among DH resources not by duplicating them in a central repository, but by linking them together via flexible APIs. It revolutionizes how scholars can work with textual sources by promoting a flexible, networked approach to digital infrastructure development. Crucially, RISE is a loosely-coupled software with flexible topologies; it can enable both federated or centralized linkages, and it can even "disappear" as long as its API and metadata standards remain in place to facilitate communications among distributed databases and tools in the back-end. Thus, unlike largescale infrastructural projects, RISE actively lowers the profile of centralized infrastructure and instead promotes existing tools and resources by enabling their interoperability in a flexible and distributed manner. As a result, it allows scholars to fully leverage the potential of material digitization and digital research tools without recreating silos of resources in the digital realm. We believe that RISE, coupled with developing novel licensing models suited for digital research methods (including consumptive research like text mining), would significantly improve the infrastructure behind DH scholarly research in sinology and beyond.
This article gives an overview of the Local Gazetteers Research Tools (LoGaRT), including its dev... more This article gives an overview of the Local Gazetteers Research Tools (LoGaRT), including its development, technical features, methodology, and examples of research applications by members of the Tu 圖 working group. The use of LoGaRT is illustrated with four brief introductions to projects that draw on visual materials from the local gazetteers, including ritual-related illustrations, city layout maps, and maps with western cartographic features. See the websites for more detailed information on LoGaRT and other research projects using it.1
Designing a protocol for the interoperability of digital textual resources—or, more simply, a “II... more Designing a protocol for the interoperability of digital textual resources—or, more simply, a “IIIF for texts”—remains a challenge, as such a protocol must cater to their vastly heterogenous formats, structures, languages, text encodings and metadata. There have been many attempts to propose a standard for textual resource interoperability, from the ubiquitous Text Encoding Initiative (TEI) format to more recent proposals like the Distributed Text Services (DTS) protocol. In this paper, we critically survey these attempts and introduce our proposal called SHINE, which aims to escape from TEI’s legacy and prioritize instead the ease for software developers to representation and exchange textual resources and their associated metadata. We do so by combining a hierarchical model of textual structure with a flexible metadata scheme in SHINE, and we continue to define and develop it based on user-centered and iterative design principles. Therefore, we argue that SHINE is a protocol for t...
Digital humanities (DH) is a burgeoning field of research in Sinology and Asian studies more broa... more Digital humanities (DH) is a burgeoning field of research in Sinology and Asian studies more broadly, and its diversity and maturity necessitate a cyberinfrastructure fit for DH-focused Sinologists’ specific needs. “Asia Network” is our solution. It is a pioneering approach for resource dissemination and emerging data analytics (such as text mining and other fair-use, consumptive research techniques) in the humanities. It is a language-agnostic software that facilitates the secure linkage between third-party research tools to different third-party textual collections (both licensed and open-access ones) via application programming interfaces (APIs). It revolutionizes how scholars can work with textual sources by promoting a flexible, networked approach to e-infrastructure development. Crucially, Asia Network is a loosely-coupled software with flexible topologies; it can enable both federated or centralized linkages, and it can even “disappear” as long as its API standards remain in ...
Land deeds were the only proof of ownership in pre-1900 Taiwan. They are indispensable for the st... more Land deeds were the only proof of ownership in pre-1900 Taiwan. They are indispensable for the studies of Taiwan's social, anthropological, and economic evolution. We have built a full-text digital library that contains almost 40,000 land deeds. The deeds in our collection range over 250 years and are collected from over 100 sources. The unprecedented volume and diversity of the sources provide an exciting source of primary documents for historians. But they also pose an interesting challenge: how to tell if two land deeds are related. In this article, we describe an approach to discover two important relations: successive transactions and allotment agreements involving the same property. Our method enabled us to construct 6,035 such transaction pairs. We also introduce a notion of 'land transitivity graph' to capture the transitivity embedded in these transactions. We discovered 2,436 such graphs, the largest of which includes 104 deeds. Some of these graphs involve land behavior that had never been studied before.
International Journal of Humanities and Arts Computing, 2012
The Qing Imperial Court documents are a major source of primary research material for studying th... more The Qing Imperial Court documents are a major source of primary research material for studying the Qing era China since they provide the most direct and first-hand details of how national affairs were handled. However, the way Qing archived these documents has made it cumbersome to collect documents covering the same event and rebuild their original contexts. In this paper, we describe some information technology that we have developed to discover two important and useful relations among these documents. The first is the citation relation among the Imperial Edicts and the Memorials. We discovered 6,801 pairs from the 37,831 Taiwan-related Imperial Court documents in the Taiwan History Digital Library (THDL) and produced 1,101 graphs of successive citations, which we call IE-M diagrams. The second relation is a template relation, which indicates groups of documents that were created following a specific format. Numerical data can also be tabulated from these documents and be used for...
As digital humanities research relies on the digitization of sources, many of its applications ar... more As digital humanities research relies on the digitization of sources, many of its applications are based on access to data on a huge scale that makes quantitative analyses and distant reading (or a birds-eye view) possible. Based on this assumption, we show how the genre of Chinese local gazetteers, with its volume, consistent structures, and broad geographic and temporal range, provides an ideal case to benefit from the digital approach. This paper introduces the Local Gazetteers Research Tools (LoGaRT), a suite of research tools designed for studying Chinese local gazetteers based on the philosophy that any comprehensive genre, such as Chinese local gazetteers, when accompanied with tools that aim to bring a collective lens to the genre, can greatly enrich the ways that scholars approach it and can transform the genre into a research infrastructure that enables new types of research. We report on how LoGaRT opens up new perspectives for researching Chinese history by showing case ...
Building temples in order to obtain relief from natural plagues was a common religious practice i... more Building temples in order to obtain relief from natural plagues was a common religious practice in premodern societies. In historical China, citizens built locust temples in hope of avoiding locust infestations. There were no centrally collected records in historical China of such plagues or other natural disasters. In order to discern patterns in the distribution of locust plagues over time and geographical space throughout historical China, this paper replicates the work of Chinese geographer Chen Chengsiang, who used local gazetteers as major sources for collecting such data. The results of this paper include a modern gis map of locust temples based on digital editions of local gazetteers, a gis dataset, and a procedural method for constructing gis maps on other topics mentioned in local gazetteers.
This paper introduces a digital humanities project at the Max Planck Institute for the History of... more This paper introduces a digital humanities project at the Max Planck Institute for the History of Science (Max-‐Planck-‐Institut für Wissenschaftsgeschichte, MPIWG) that aims to unlock the treasure chest of local knowledge written in the genre of Chinese local gazetteers for computer assisted analyses. In the past two decades, a great amount of historical documents have been digitized and put on the Web to enable easy access for scholars around the globe. In parallel, the amount of searchable full-‐text versions of historical texts has increased, which opens the possibility of text mining the contents for large scaled analyses. Many works in this direction have been proposed and got recognition, while they are also criticized for drawing conclusions from seemingly imprecise results due to the restriction that their algorithms have no knowledge about the meanings of the different pieces in a text (Jockers, 2013; Google Ngram Viewer; Chen et al., 2007). An alternative approach is thus to first " teach " computers what each pieces of a text means before asking computers to run automatic analyses. Such " teaching " is done by tagging, or called markup. Many digital humanities projects have been using TEI, a standard for text encoding based on XML, to tag their research materials (TEI; Flanders). In the Local Gazetteers Project, since the genre organizes knowledge in a very structural way, we are also using tagging to teach computers the meanings of texts in order to turn them into data tables to enable computer assisted analyses including GIS mapping. However, since the amount of texts is huge, we also propose a research data repository for scholars to collaborate in this project and to aggregate their results. What are the Chinese Local Gazetteers? The Chinese local gazetteers is a genre of texts that has been produced in China consistently from the 10 th century on to even today. Most of them are compiled by local officials as a major means to collect and aggregate historical, social, and geographical knowledge of an administrative region for governing purposes. There are at least 8,000 titles of pre-‐1949 local gazetteers still extant today. They cover almost every well-‐populated region in historical China. Despite being compiled by different officials for different regions, the local gazetteers have developed a pretty consistent structure of " describing " local knowledge. Most gazetteers contain the following chapters: history,
These are the slides (in Chinese) that I used for the Digital Humanities workshop organized by Pe... more These are the slides (in Chinese) that I used for the Digital Humanities workshop organized by Peking University on May 23, 2020. In this online seminar, I talked about information technologies and historical documents analyses in general, and introduced LoGaRT (Local Gazetteers Research Tools) and showed how these methods were applied to the Chinese local gazetteers (difangzhi 地方志).
Uploads
Papers by Shih-Pei Chen