This deliverable covers the description and production of a semantically annotated corpus. This i... more This deliverable covers the description and production of a semantically annotated corpus. This is available within the Sekt consortium as training and test data for the machine learning algorithms for semantic annotation and as a gold standard for the evaluation of techniques.
7th Language Resources and Evaluation …, Jan 1, 2010
In this paper, we present an approach to measure the transliteration similarity of English-Hindi ... more In this paper, we present an approach to measure the transliteration similarity of English-Hindi word pairs. Our approach has two components. First we propose a bi-directional mapping between one or more characters in the Devanagari script and one or more characters in the Roman script (pronounced as in English). This allows a given Hindi word written in Devanagari to be transliterated into the Roman script and vice-versa. Second, we present an algorithm for computing a similarity measure that is a variant of Dice's coefficient measure and the LCSR measure and which also takes into account the constraints needed to match English-Hindi transliterated words. Finally, by evaluating various similarity metrics individually and together under a multiple measure agreement scenario, we show that it is possible to achieve a 0.92 f-measure in identifying English-Hindi word pairs that are transliterations. In order to assess the portability of our approach to other similar languages we adapt our system to the Gujarati language.
EU-IST Integrated Project (IP) IST-2003-506826 SEKT Deliverable D6.6.4 (WP6) This deliverable is ... more EU-IST Integrated Project (IP) IST-2003-506826 SEKT Deliverable D6.6.4 (WP6) This deliverable is a part of a series of bottom-up deliverables D6.6.x on bilateral integration activities and provides information on the integration of GATE and KAON2 tools.
7th Language Resources and Evaluation …, Jan 1, 2010
A considerable amount of work has been put into development of stemmers and morphological analyse... more A considerable amount of work has been put into development of stemmers and morphological analysers. The majority of these approaches use hand-crafted suffix-replacement rules but a few try to discover such rules from corpora. While most of the approaches remove or replace suffixes, there are examples of derivational stemmers which are based on prefixes as well. In this paper we present a rule-based morphological analyser. We propose an approach that takes both prefixes as well as suffixes into account. Given a corpus and a dictionary, our method can be used to obtain a set of suffix-replacement rules for deriving an inflected word's root form. We developed an approach for the Hindi language but show that the approach is portable, at least to related languages, by adapting it to the Gujarati language. Given that the entire process of developing such a ruleset is simple and fast, our approach can be used for rapid development of morphological analysers and yet it can obtain competitive results with analysers built relying on human authored rules.
... A framework for ontology learning and data-driven change discovery. ... In Proceedings of the... more ... A framework for ontology learning and data-driven change discovery. ... In Proceedings of the AtlanticWeb Intelligence Conference, AWIC, pages 380-386. ... 59. Nancy Ide , Jean Véronis, Introduction to the special issue on word sense disambiguation: the state of the art ...
Executive Summary The Large Knowledge Collider (LarKC) project is building a platform for scaleab... more Executive Summary The Large Knowledge Collider (LarKC) project is building a platform for scaleable reasoning over terabytes of scientific data, using massive distributed incomplete reasoning. One of the use cases is carcinogenesis research. This has two scenarios, as ...
Executive Summary The Large Knowledge Collider (LarKC) project is building a platform for scaleab... more Executive Summary The Large Knowledge Collider (LarKC) project is building a platform for scaleable reasoning over terabytes of scientific data, using massive distributed incomplete reasoning. One of the use cases is carcinogenesis research, as described in LarKC ...
... Extraction and Semantic Annotation for Multi-Paradigm Information Management}}, booktitle = {... more ... Extraction and Semantic Annotation for Multi-Paradigm Information Management}}, booktitle = {{Current Challenges in Patent Information Retrieval}}, publisher = {Springer}, series = {The Information Retrieval Series}, volume = 29, editor = {Mihai Lupu and Katja Mayer and John ...
When researching new product ideas or filing new patents, inventors need to retrieve all relevant... more When researching new product ideas or filing new patents, inventors need to retrieve all relevant pre-existing know-how and/or to exploit and enforce patents in their technological domain. However, this process is hindered by lack of richer metadata, which if present, would allow more powerful concept-based search to complement the current keywordbased approach. This paper presents our approach to automatic patent enrichment, tested in large-scale, parallel experiments on USPTO and EPO documents. It starts by defining the metadata annotation task and examines its challenges. The text analysis tools are presented next, including details on automatic annotation of sections, references and measurements. The key challenges encountered were dealing with ambiguities and errors in the data; creation and maintenance of large, domain-independent dictionaries; and building an efficient, robust patent analysis pipeline, capable of dealing with terabytes of data. The accuracy of automatically created metadata is evaluated against a human-annotated gold standard, with results of over 90% on most annotation types.
In this paper, we describe a word alignment algorithm for English-Hindi parallel data. The system... more In this paper, we describe a word alignment algorithm for English-Hindi parallel data. The system was developed to participate in the shared task on word alignment for languages with scarce resources at the ACL 2005 workshop, on "Building and using parallel texts: data driven machine translation and beyond". Our word alignment algorithm is based on a hybrid method which performs local word grouping on Hindi sentences and uses other methods such as dictionary lookup, transliteration similarity, expected English words and nearest aligned neighbours. We trained our system on the training data provided to obtain a list of named entities and cognates and to collect rules for local word grouping in Hindi sentences. The system scored 77.03% precision and 60.68% recall on the shared task unseen test data.
The need for efficient corpus indexing and querying arises frequently both in machine learning-ba... more The need for efficient corpus indexing and querying arises frequently both in machine learning-based and human-engineered natural language processing systems.
This deliverable covers the description and production of a semantically annotated corpus. This i... more This deliverable covers the description and production of a semantically annotated corpus. This is available within the Sekt consortium as training and test data for the machine learning algorithms for semantic annotation and as a gold standard for the evaluation of techniques.
7th Language Resources and Evaluation …, Jan 1, 2010
In this paper, we present an approach to measure the transliteration similarity of English-Hindi ... more In this paper, we present an approach to measure the transliteration similarity of English-Hindi word pairs. Our approach has two components. First we propose a bi-directional mapping between one or more characters in the Devanagari script and one or more characters in the Roman script (pronounced as in English). This allows a given Hindi word written in Devanagari to be transliterated into the Roman script and vice-versa. Second, we present an algorithm for computing a similarity measure that is a variant of Dice's coefficient measure and the LCSR measure and which also takes into account the constraints needed to match English-Hindi transliterated words. Finally, by evaluating various similarity metrics individually and together under a multiple measure agreement scenario, we show that it is possible to achieve a 0.92 f-measure in identifying English-Hindi word pairs that are transliterations. In order to assess the portability of our approach to other similar languages we adapt our system to the Gujarati language.
EU-IST Integrated Project (IP) IST-2003-506826 SEKT Deliverable D6.6.4 (WP6) This deliverable is ... more EU-IST Integrated Project (IP) IST-2003-506826 SEKT Deliverable D6.6.4 (WP6) This deliverable is a part of a series of bottom-up deliverables D6.6.x on bilateral integration activities and provides information on the integration of GATE and KAON2 tools.
7th Language Resources and Evaluation …, Jan 1, 2010
A considerable amount of work has been put into development of stemmers and morphological analyse... more A considerable amount of work has been put into development of stemmers and morphological analysers. The majority of these approaches use hand-crafted suffix-replacement rules but a few try to discover such rules from corpora. While most of the approaches remove or replace suffixes, there are examples of derivational stemmers which are based on prefixes as well. In this paper we present a rule-based morphological analyser. We propose an approach that takes both prefixes as well as suffixes into account. Given a corpus and a dictionary, our method can be used to obtain a set of suffix-replacement rules for deriving an inflected word's root form. We developed an approach for the Hindi language but show that the approach is portable, at least to related languages, by adapting it to the Gujarati language. Given that the entire process of developing such a ruleset is simple and fast, our approach can be used for rapid development of morphological analysers and yet it can obtain competitive results with analysers built relying on human authored rules.
... A framework for ontology learning and data-driven change discovery. ... In Proceedings of the... more ... A framework for ontology learning and data-driven change discovery. ... In Proceedings of the AtlanticWeb Intelligence Conference, AWIC, pages 380-386. ... 59. Nancy Ide , Jean Véronis, Introduction to the special issue on word sense disambiguation: the state of the art ...
Executive Summary The Large Knowledge Collider (LarKC) project is building a platform for scaleab... more Executive Summary The Large Knowledge Collider (LarKC) project is building a platform for scaleable reasoning over terabytes of scientific data, using massive distributed incomplete reasoning. One of the use cases is carcinogenesis research. This has two scenarios, as ...
Executive Summary The Large Knowledge Collider (LarKC) project is building a platform for scaleab... more Executive Summary The Large Knowledge Collider (LarKC) project is building a platform for scaleable reasoning over terabytes of scientific data, using massive distributed incomplete reasoning. One of the use cases is carcinogenesis research, as described in LarKC ...
... Extraction and Semantic Annotation for Multi-Paradigm Information Management}}, booktitle = {... more ... Extraction and Semantic Annotation for Multi-Paradigm Information Management}}, booktitle = {{Current Challenges in Patent Information Retrieval}}, publisher = {Springer}, series = {The Information Retrieval Series}, volume = 29, editor = {Mihai Lupu and Katja Mayer and John ...
When researching new product ideas or filing new patents, inventors need to retrieve all relevant... more When researching new product ideas or filing new patents, inventors need to retrieve all relevant pre-existing know-how and/or to exploit and enforce patents in their technological domain. However, this process is hindered by lack of richer metadata, which if present, would allow more powerful concept-based search to complement the current keywordbased approach. This paper presents our approach to automatic patent enrichment, tested in large-scale, parallel experiments on USPTO and EPO documents. It starts by defining the metadata annotation task and examines its challenges. The text analysis tools are presented next, including details on automatic annotation of sections, references and measurements. The key challenges encountered were dealing with ambiguities and errors in the data; creation and maintenance of large, domain-independent dictionaries; and building an efficient, robust patent analysis pipeline, capable of dealing with terabytes of data. The accuracy of automatically created metadata is evaluated against a human-annotated gold standard, with results of over 90% on most annotation types.
In this paper, we describe a word alignment algorithm for English-Hindi parallel data. The system... more In this paper, we describe a word alignment algorithm for English-Hindi parallel data. The system was developed to participate in the shared task on word alignment for languages with scarce resources at the ACL 2005 workshop, on "Building and using parallel texts: data driven machine translation and beyond". Our word alignment algorithm is based on a hybrid method which performs local word grouping on Hindi sentences and uses other methods such as dictionary lookup, transliteration similarity, expected English words and nearest aligned neighbours. We trained our system on the training data provided to obtain a list of named entities and cognates and to collect rules for local word grouping in Hindi sentences. The system scored 77.03% precision and 60.68% recall on the shared task unseen test data.
The need for efficient corpus indexing and querying arises frequently both in machine learning-ba... more The need for efficient corpus indexing and querying arises frequently both in machine learning-based and human-engineered natural language processing systems.
Uploads
Papers by Niraj Aswani