Lexical Ambiguity Resolution for Turkish in Direct Transfer Machine Translation Models
Lecture Notes in Computer Science, 2006
... Lexical Ambiguity Resolution for Turkish in Direct Transfer Machine Translation Models A. Cün... more ... Lexical Ambiguity Resolution for Turkish in Direct Transfer Machine Translation Models A. Cüneyd Tantuğ1, Eşref Adalı1, and Kemal Oflazer2 ... In this case, the baseline translation model outperforms all other LMs. Input : Hamyr gyzgyn tamdyrda esli wagtlap durandan ...
Using finite state technology in a tool for linguistic exploration
Intelligent, interactive and pervasively accessible tools for providing information about element... more Intelligent, interactive and pervasively accessible tools for providing information about elements of a language are crucial in learning a language, especially in an advanced secondary language learning setting and learning for linguistic exploration. The paper describes a prototype implementation of a tool that provides intelligent, active and interactive tools for helping linguistics students inquire and learn about lexical and syntactic
MISE, Machine for In-System Evaluation of custom VLSI chips
This paper identifies some of the key research problems that one encounters in specifying, design... more This paper identifies some of the key research problems that one encounters in specifying, designing, testing and demonstrating a custom chip in relation to the application system in which it will be used, and proposes a system called MISE (Machine For In-System Evaluation) to be a solution to the issues raised.
... 5.1 Issues Related to the Differences between English and Turkish Tense differences. There ar... more ... 5.1 Issues Related to the Differences between English and Turkish Tense differences. There are some differences between tenses in Turkish and English. Some English tenses do not have exactTurkish counterparts, and vice-Page 8. 90 DZ Hakkani et al. versa. ...
BALKANET A Multilingual Semantic Network for the Balkan Languages
BalkaNet aims at building a multilingual lexical database consisting of WordNets in several Centr... more BalkaNet aims at building a multilingual lexical database consisting of WordNets in several Central and Eastern European languages. Even though it will be built in a similar way with EuroWordNet, new features will be implemented ranging from structuring the Inter-Lingual- Index to ensure linking of conceptual equivalencies across WordNets to the development of an inter-networked WordNet Management so that each partner retains full responsibility and independence of his local WordNet whereas at the same time they will be able to view other WordNets and check their compatibility.
Parsing Agglutinative Word Structures And Its Application To Spelling Checking For Turkish
International Conference on Computational Linguistics, 1992
Most of the research on parsing natural languages has been concerned with English, or with other ... more Most of the research on parsing natural languages has been concerned with English, or with other languages morphologically similar to English. Parsing agglutinative word structures has attracted relatively little attention most probably because agglutinative languages contain word structures of considerable complexity, and parsing words in such languages requires morphological analysis techniques. In this paper, we present the design and implementation
Meeting of the Association for Computational Linguistics, 1997
We present a constraint-based morphological disambiguation system in which individual constraints... more We present a constraint-based morphological disambiguation system in which individual constraints vote on matching morphological parses, and disambiguation of all the tokens in a sentence is performed at the end by selecting parses that receive the highest votes. This constraint application paradigm makes the outcome of the disambiguation . Our system is implemented in Prolog and we are currently investigating
This paper presents a constraint-based morphological disambiguation approach that is applicable l... more This paper presents a constraint-based morphological disambiguation approach that is applicable languages with complex morphology{speci cally agglutinative languages with productive in ectional and derivational morphological phenomena. In certain respects, our approach has been motivated by Brill's recent work , but with the observation that his transformational approach is not directly applicable to languages like Turkish. Our system combines corpus independent handcrafted constraint rules, constraint rules that are learned via unsupervised learning from a training corpus, and additional statistical information from the corpus to be morphologically disambiguated. The hand-crafted rules are linguistically motivated and tuned to improve precision without sacri cing recall. The unsupervised learning process produces two sets of rules: (i) choose rules which choose morphological parses of a lexical item satisfying constraint e ectively discarding other parses, and (ii) delete rules, which delete parses satisfying a constraint. Our approach also uses a novel approach to unknown word processing by employing a secondary morphological processor which recovers any relevant in ectional and derivational information from a lexical item whose root is unknown. With this approach, well below 1% of the tokens remains as unknown in the texts we have experimented with. Our results indicate that by combining these hand-crafted, statistical and learned information sources, we can attain a recall of 96 to 97% with a corresponding precision of 93 to 94%, and ambiguity of 1.02 to 1.03 parses per token.
Proceedings of the International Workshop on Finite State Methods in Natural Language Processing - FSMNLP '09, 1998
We describe a constraint-based morphological disambiguation system in which individual constraint... more We describe a constraint-based morphological disambiguation system in which individual constraint rules vote on matching morphological parses followed by its implementation using finite state transducers. Voting constraint rules have a number of desirable properties: The outcome of the disambiguation is independent of the order of application of the local contextual constraint rules. Thus the rule developer is relieved from worrying about conflicting rule sequencing. The approach can also combine statistically and manually obtained constraints, and incorporate negative constraints that rule out certain patterns. The transducer implementation has a number of desirable properties compared to other finite state tagging and light parsing approaches, implemented with automata intersection. The most important of these is that since constraints do not remove parses there is no risk of an overzealous constraint "killing a sentence ~ by removing all parses of a token during intersection. After a description of our approach we present preliminary results from tagging the Wall Street Journal Corpus with this approach. With about 400 statistically derived constraints and about 570 manual constraints, we can attain an accuracy of 97.82% on the training corpus and 97.29% on the test corpus. We then describe a finite state implementation of our approach and discuss various related issues.
Proceedings of the 17th international conference on Computational linguistics -, 1998
... bilkent, edu. tr Abstract: We describe a constraint-based tagging approach where individual c... more ... bilkent, edu. tr Abstract: We describe a constraint-based tagging approach where individual constraint rules vote on sequences of matching tokens and tags. Disambiguation of all tokens in a sentence is performed at the very ...
We present a progress report of the Turkish Treebank concentrating on vari- ous aspects of its de... more We present a progress report of the Turkish Treebank concentrating on vari- ous aspects of its design and implemen- tation. In addition to a review of the cor- pus compilation process and the design of the annotation scheme, we describe the details of various pre-processing stages and the computer-assisted anno- tation process.
Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06, 2006
This paper investigates the use of sublexical units as a solution to handling the complex morphol... more This paper investigates the use of sublexical units as a solution to handling the complex morphology with productive derivational processes, in the development of a lexical functional grammar for Turkish. Such sublexical units make it possible to expose the internal structure of words with multiple derivations to the grammar rules in a uniform manner. This in turn leads to more succinct and manageable rules. Further, the semantics of the derivations can also be systematically reflected in a compositional way by constructing PRED values on the fly. We illustrate how we use sublexical units for handling simple productive derivational morphology and more interesting cases such as causativization, etc., which change verb valency. Our priority is to handle several linguistic phenomena in order to observe the effects of our approach on both the c-structure and the f-structure representation, and grammar writing, leaving the coverage and evaluation issues aside for the moment.
Proceedings of the Workshop on Multiword Expressions Integrating Processing - MWE '04, 2004
This paper describes a multi-word expression processor for preprocessing Turkish text for various... more This paper describes a multi-word expression processor for preprocessing Turkish text for various language engineering applications. In addition to the fairly standard set of lexicalized collocations and multi-word expressions such as named-entities, Turkish uses a quite wide range of semi-lexicalized and non-lexicalized collocations. After an overview of relevant aspects of Turkish, we present a description of the multi-word expressions we handle. We then summarize the computational setting in which we employ a series of components for tokenization, morphological analysis, and multi-word expression extraction. We finally present results from runs over a large corpus and a small gold-standard corpus.
Efficient Find-and-Replace in Agglutinative Languages: The Case of Turkish
The Case of Turkish. Orhan Bilgin, Özlem Çetinoğlu, Kemal Oflazer. {orhanb, ozlemc, oflazer}@saba... more The Case of Turkish. Orhan Bilgin, Özlem Çetinoğlu, Kemal Oflazer. {orhanb, ozlemc, oflazer}@sabanciuniv.edu. ... Hence, gördü mü? (did he see?) becomes gitti mi? (did he go?) andAli de (also Ali) becomes Murat da (also Murat). These postpositions may also follow each other: ...
ABSTRACT The daily spoken variety of Arabic is often termed the colloquial or dialect form of Ara... more ABSTRACT The daily spoken variety of Arabic is often termed the colloquial or dialect form of Arabic. There are many Arabic dialects across the Arab World and within other Arabic speaking communities. These dialects vary widely from region to region and to a lesser extent from city to city in each region. The dialects are not standardized, they are not taught, and they do not have official status. However they are the primary vehicles of communication (face-to-face and recently, online) and have a large presence in the arts as well. In this paper, we present the first multidialectal Arabic parallel corpus, a collection of 2,000 sentences in Standard Arabic, Egyptian, Tunisian, Jordanian, Palestinian and Syrian Arabic, in addition to English. Such parallel data does not exist naturally, which makes this corpus a very valuable resource that has many potential applications such as Arabic dialect identification and machine translation.
YouDACC: the Youtube Dialectal Arabic Commentary Corpus
ABSTRACT In the Arab world, while Modern Standard Arabic is commonly used in formal written conte... more ABSTRACT In the Arab world, while Modern Standard Arabic is commonly used in formal written context, on sites like Youtube, people are increasingly using Dialectal Arabic, the language for everyday use to comment on a video and interact with the community. These user-contributed comments along with the video and user attributes, offer a rich source of multi-dialectal Arabic sentences and expressions from different countries in the Arab world. This paper presents YOUDACC, an automatically annotated large-scale multi-dialectal Arabic corpus collected from user comments on Youtube videos. Our corpus covers different groups of dialects: Egyptian (EG), Gulf (GU), Iraqi (IQ), Maghrebi (MG) and Levantine (LV). We perform an empirical analysis on the crawled corpus and demonstrate that our location-based proposed method is effective for the task of dialect labeling.
Lexical Ambiguity Resolution for Turkish in Direct Transfer Machine Translation Models
Lecture Notes in Computer Science, 2006
... Lexical Ambiguity Resolution for Turkish in Direct Transfer Machine Translation Models A. Cün... more ... Lexical Ambiguity Resolution for Turkish in Direct Transfer Machine Translation Models A. Cüneyd Tantuğ1, Eşref Adalı1, and Kemal Oflazer2 ... In this case, the baseline translation model outperforms all other LMs. Input : Hamyr gyzgyn tamdyrda esli wagtlap durandan ...
Using finite state technology in a tool for linguistic exploration
Intelligent, interactive and pervasively accessible tools for providing information about element... more Intelligent, interactive and pervasively accessible tools for providing information about elements of a language are crucial in learning a language, especially in an advanced secondary language learning setting and learning for linguistic exploration. The paper describes a prototype implementation of a tool that provides intelligent, active and interactive tools for helping linguistics students inquire and learn about lexical and syntactic
MISE, Machine for In-System Evaluation of custom VLSI chips
This paper identifies some of the key research problems that one encounters in specifying, design... more This paper identifies some of the key research problems that one encounters in specifying, designing, testing and demonstrating a custom chip in relation to the application system in which it will be used, and proposes a system called MISE (Machine For In-System Evaluation) to be a solution to the issues raised.
... 5.1 Issues Related to the Differences between English and Turkish Tense differences. There ar... more ... 5.1 Issues Related to the Differences between English and Turkish Tense differences. There are some differences between tenses in Turkish and English. Some English tenses do not have exactTurkish counterparts, and vice-Page 8. 90 DZ Hakkani et al. versa. ...
BALKANET A Multilingual Semantic Network for the Balkan Languages
BalkaNet aims at building a multilingual lexical database consisting of WordNets in several Centr... more BalkaNet aims at building a multilingual lexical database consisting of WordNets in several Central and Eastern European languages. Even though it will be built in a similar way with EuroWordNet, new features will be implemented ranging from structuring the Inter-Lingual- Index to ensure linking of conceptual equivalencies across WordNets to the development of an inter-networked WordNet Management so that each partner retains full responsibility and independence of his local WordNet whereas at the same time they will be able to view other WordNets and check their compatibility.
Parsing Agglutinative Word Structures And Its Application To Spelling Checking For Turkish
International Conference on Computational Linguistics, 1992
Most of the research on parsing natural languages has been concerned with English, or with other ... more Most of the research on parsing natural languages has been concerned with English, or with other languages morphologically similar to English. Parsing agglutinative word structures has attracted relatively little attention most probably because agglutinative languages contain word structures of considerable complexity, and parsing words in such languages requires morphological analysis techniques. In this paper, we present the design and implementation
Meeting of the Association for Computational Linguistics, 1997
We present a constraint-based morphological disambiguation system in which individual constraints... more We present a constraint-based morphological disambiguation system in which individual constraints vote on matching morphological parses, and disambiguation of all the tokens in a sentence is performed at the end by selecting parses that receive the highest votes. This constraint application paradigm makes the outcome of the disambiguation . Our system is implemented in Prolog and we are currently investigating
This paper presents a constraint-based morphological disambiguation approach that is applicable l... more This paper presents a constraint-based morphological disambiguation approach that is applicable languages with complex morphology{speci cally agglutinative languages with productive in ectional and derivational morphological phenomena. In certain respects, our approach has been motivated by Brill's recent work , but with the observation that his transformational approach is not directly applicable to languages like Turkish. Our system combines corpus independent handcrafted constraint rules, constraint rules that are learned via unsupervised learning from a training corpus, and additional statistical information from the corpus to be morphologically disambiguated. The hand-crafted rules are linguistically motivated and tuned to improve precision without sacri cing recall. The unsupervised learning process produces two sets of rules: (i) choose rules which choose morphological parses of a lexical item satisfying constraint e ectively discarding other parses, and (ii) delete rules, which delete parses satisfying a constraint. Our approach also uses a novel approach to unknown word processing by employing a secondary morphological processor which recovers any relevant in ectional and derivational information from a lexical item whose root is unknown. With this approach, well below 1% of the tokens remains as unknown in the texts we have experimented with. Our results indicate that by combining these hand-crafted, statistical and learned information sources, we can attain a recall of 96 to 97% with a corresponding precision of 93 to 94%, and ambiguity of 1.02 to 1.03 parses per token.
Proceedings of the International Workshop on Finite State Methods in Natural Language Processing - FSMNLP '09, 1998
We describe a constraint-based morphological disambiguation system in which individual constraint... more We describe a constraint-based morphological disambiguation system in which individual constraint rules vote on matching morphological parses followed by its implementation using finite state transducers. Voting constraint rules have a number of desirable properties: The outcome of the disambiguation is independent of the order of application of the local contextual constraint rules. Thus the rule developer is relieved from worrying about conflicting rule sequencing. The approach can also combine statistically and manually obtained constraints, and incorporate negative constraints that rule out certain patterns. The transducer implementation has a number of desirable properties compared to other finite state tagging and light parsing approaches, implemented with automata intersection. The most important of these is that since constraints do not remove parses there is no risk of an overzealous constraint "killing a sentence ~ by removing all parses of a token during intersection. After a description of our approach we present preliminary results from tagging the Wall Street Journal Corpus with this approach. With about 400 statistically derived constraints and about 570 manual constraints, we can attain an accuracy of 97.82% on the training corpus and 97.29% on the test corpus. We then describe a finite state implementation of our approach and discuss various related issues.
Proceedings of the 17th international conference on Computational linguistics -, 1998
... bilkent, edu. tr Abstract: We describe a constraint-based tagging approach where individual c... more ... bilkent, edu. tr Abstract: We describe a constraint-based tagging approach where individual constraint rules vote on sequences of matching tokens and tags. Disambiguation of all tokens in a sentence is performed at the very ...
We present a progress report of the Turkish Treebank concentrating on vari- ous aspects of its de... more We present a progress report of the Turkish Treebank concentrating on vari- ous aspects of its design and implemen- tation. In addition to a review of the cor- pus compilation process and the design of the annotation scheme, we describe the details of various pre-processing stages and the computer-assisted anno- tation process.
Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06, 2006
This paper investigates the use of sublexical units as a solution to handling the complex morphol... more This paper investigates the use of sublexical units as a solution to handling the complex morphology with productive derivational processes, in the development of a lexical functional grammar for Turkish. Such sublexical units make it possible to expose the internal structure of words with multiple derivations to the grammar rules in a uniform manner. This in turn leads to more succinct and manageable rules. Further, the semantics of the derivations can also be systematically reflected in a compositional way by constructing PRED values on the fly. We illustrate how we use sublexical units for handling simple productive derivational morphology and more interesting cases such as causativization, etc., which change verb valency. Our priority is to handle several linguistic phenomena in order to observe the effects of our approach on both the c-structure and the f-structure representation, and grammar writing, leaving the coverage and evaluation issues aside for the moment.
Proceedings of the Workshop on Multiword Expressions Integrating Processing - MWE '04, 2004
This paper describes a multi-word expression processor for preprocessing Turkish text for various... more This paper describes a multi-word expression processor for preprocessing Turkish text for various language engineering applications. In addition to the fairly standard set of lexicalized collocations and multi-word expressions such as named-entities, Turkish uses a quite wide range of semi-lexicalized and non-lexicalized collocations. After an overview of relevant aspects of Turkish, we present a description of the multi-word expressions we handle. We then summarize the computational setting in which we employ a series of components for tokenization, morphological analysis, and multi-word expression extraction. We finally present results from runs over a large corpus and a small gold-standard corpus.
Efficient Find-and-Replace in Agglutinative Languages: The Case of Turkish
The Case of Turkish. Orhan Bilgin, Özlem Çetinoğlu, Kemal Oflazer. {orhanb, ozlemc, oflazer}@saba... more The Case of Turkish. Orhan Bilgin, Özlem Çetinoğlu, Kemal Oflazer. {orhanb, ozlemc, oflazer}@sabanciuniv.edu. ... Hence, gördü mü? (did he see?) becomes gitti mi? (did he go?) andAli de (also Ali) becomes Murat da (also Murat). These postpositions may also follow each other: ...
ABSTRACT The daily spoken variety of Arabic is often termed the colloquial or dialect form of Ara... more ABSTRACT The daily spoken variety of Arabic is often termed the colloquial or dialect form of Arabic. There are many Arabic dialects across the Arab World and within other Arabic speaking communities. These dialects vary widely from region to region and to a lesser extent from city to city in each region. The dialects are not standardized, they are not taught, and they do not have official status. However they are the primary vehicles of communication (face-to-face and recently, online) and have a large presence in the arts as well. In this paper, we present the first multidialectal Arabic parallel corpus, a collection of 2,000 sentences in Standard Arabic, Egyptian, Tunisian, Jordanian, Palestinian and Syrian Arabic, in addition to English. Such parallel data does not exist naturally, which makes this corpus a very valuable resource that has many potential applications such as Arabic dialect identification and machine translation.
YouDACC: the Youtube Dialectal Arabic Commentary Corpus
ABSTRACT In the Arab world, while Modern Standard Arabic is commonly used in formal written conte... more ABSTRACT In the Arab world, while Modern Standard Arabic is commonly used in formal written context, on sites like Youtube, people are increasingly using Dialectal Arabic, the language for everyday use to comment on a video and interact with the community. These user-contributed comments along with the video and user attributes, offer a rich source of multi-dialectal Arabic sentences and expressions from different countries in the Arab world. This paper presents YOUDACC, an automatically annotated large-scale multi-dialectal Arabic corpus collected from user comments on Youtube videos. Our corpus covers different groups of dialects: Egyptian (EG), Gulf (GU), Iraqi (IQ), Maghrebi (MG) and Levantine (LV). We perform an empirical analysis on the crawled corpus and demonstrate that our location-based proposed method is effective for the task of dialect labeling.
Uploads
Papers by Kemal Oflazer