Abstract This paper overviews FBK's participation in the Main and KBP Va... more Abstract This paper overviews FBK's participation in the Main and KBP Validation Pilot task organized within the RTE6 Evaluation Campaign. Our submissions have been produced running the EDITS (Edit Distance Textual Entailment Suite) open source RTE package, ...
This paper presents a general overview of answer-driven reasoning, which may serve as a common fr... more This paper presents a general overview of answer-driven reasoning, which may serve as a common framework for a variety of Information Retrieval related activities. The underlying assumption is that a piece of information in a given document is as relevant as the extent to which such information represents the answer to a possible user's question. The approach relies on the application of answer procedures to a source data (e.g. a text collection) to mine question/answer correlations. The outcome can be exploited for different purposes, ranging from the creation of a Q/A index specifically targeted to intelligent Information Retrieval, Question Answering and Summarization, to the augmentation of existing ontologies in the Semantic Web scenario.
We address a core aspect of the multilingual content synchronization task: the identifica-tion of... more We address a core aspect of the multilingual content synchronization task: the identifica-tion of novel, more informative or semanti-cally equivalent pieces of information in two documents about the same topic. This can be seen as an application-oriented variant of tex-tual entailment recognition where: i) T and H are in different languages, and ii) entail-ment relations between T and H have to be checked in both directions. Using a combi-nation of lexical, syntactic, and semantic fea-tures to train a cross-lingual textual entailment system, we report promising results on differ-ent datasets.
We address two challenges for automatic machine translation evaluation: a) avoiding the use of re... more We address two challenges for automatic machine translation evaluation: a) avoiding the use of reference translations, and b) focusing on adequacy estimation. From an economic perspective, getting rid of costly hand-crafted reference translations (a) permits to alleviate the main bottleneck in MT evaluation. From a system evaluation perspective, pushing semantics into MT (b) is a necessity in order to complement the shallow methods currently used overcoming their limitations. Casting the problem as a cross-lingual textual entailment application, we experiment with different benchmarks and evaluation settings. Our method shows high correlation with human judgements and good results on all datasets without relying on reference translations.
We address two issues related to the development of systems for Recognizing Textual Entailment. T... more We address two issues related to the development of systems for Recognizing Textual Entailment. The first is the impossibility to capitalize on lessons learned over the different datasets available, due to the changing nature of traditional RTE evaluation settings. The second is the lack of simple ways to assess the results achieved by our system on a given training corpus, and figure out its real potential on unseen test data. Our contribution is the extension of an open-source RTE package with an automatic way to explore the large search space of possible configurations, in order to select the most promising one over a given dataset. From the developers' point of view, the efficiency and ease of use of the system, together with the good results achieved on all previous RTE datasets, represent a useful support, providing an immediate term of comparison to position the results of their approach.
This paper overviews FBK's participation in the Cross-Lingual Textual Entailment for Content ... more This paper overviews FBK's participation in the Cross-Lingual Textual Entailment for Content Synchronization task organized within SemEval-2012. Our participation is characterized by using cross-lingual matching features extracted from lexical and semantic phrase tables and dependency relations. The features are used for multi-class and binary classification using SVMs. Using a combination of lexical, syntactic, and semantic features to create a cross-lingual textual entailment system, we report on experiments over the provided dataset. Our best run achieved an accuracy of 50.4% on the Spanish-English dataset (with the average score and the median system respectively achieving 40.7% and 34.6%), demonstrating the effectiveness of a "pure" cross-lingual approach that avoids intermediate translations.
This paper describes the participation of FBK in the Semantic Textual Similarity (STS) task organ... more This paper describes the participation of FBK in the Semantic Textual Similarity (STS) task organized within Semeval 2012. Our approach explores lexical, syntactic and semantic machine translation evaluation metrics combined with distributional and knowledge-based word similarity metrics. Our best model achieves 60.77% correlation with human judgements (Mean score) and ranked 20 out of 88 submitted runs in the Mean ranking, where the average correlation across all the sub-portions of the test set is considered.
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), 2015
The capability of handling anaphora is becoming a key feature for Question Answering systems, as ... more The capability of handling anaphora is becoming a key feature for Question Answering systems, as it can play a crucial role at different stages of the QA loop. At the question processing stage, on which this paper is focused, it enhances the treatment of follow-up questions, allowing for a more natural interaction with the user. As much as the QA task evolves towards a realistic dialogue-based scenario, one of the concrete problems raised by follow-up questions is tracking their actual referent. Each question may in fact refer to the topic of the session, to an answer given to an earlier question, or to a new entity it introduces in the dialogue. Focusing on the referent traceability problem, we present and experiment with a possible data-driven solution which exploits simple features of the input question and its surrounding context (the target of the session, and the previous questions) to inform the next phases of the QA process.
Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02, 2001
ABSTRACT Answer Validation is an emerging topic in Question Answering, where open domain systems ... more ABSTRACT Answer Validation is an emerging topic in Question Answering, where open domain systems are often required to rank huge amounts of candidate answers. We present a novel approach to answer validation based on the intuition that the amount of implicit knowledge which connects an answer to a question can be quantitatively estimated by exploiting the redundancy of Web information. Experiments carried out on the TREC-2001 judged-answer collection show that the approach achieves a high level of performance (i.e. 81% success rate). The simplicity and the efficiency of this approach make it suitable to be used as a module in Question Answering systems.
Proceedings of the Eighth Annual International Symposium on Wikis and Open Collaboration - WikiSym '12, 2012
ABSTRACT CoSyne is a content synchronization system for assisting users and organizations involve... more ABSTRACT CoSyne is a content synchronization system for assisting users and organizations involved in the maintenance of multilingual wikis. The system allows users to explore the diversity of multilingual content using a monolingual view. It provides suggestions for content modification based on additional or more specific information found in other language versions, and enables seamless integration of automatically translated sentences while giving users the flexibility to edit, correct and control eventual changes to the wiki page. To support these tasks, CoSyne employs state-of-the-art machine translation and natural language processing techniques.
Proceedings of the 7th International Symposium on Wikis and Open Collaboration - WikiSym '11, 2011
Wikis allow a large base of contributors easy access to shared content, and freedom in editing it... more Wikis allow a large base of contributors easy access to shared content, and freedom in editing it. One of the side-effects of this freedom was the emergence of parallel and independently evolving versions in a variety of languages, reflecting the multilingual background of the pool of contributors. For the Wiki to properly represent the user-added content, this should be fully available in all its languages. Working on parallel Wikis in several European languages, we investigate the possibility to "synchronize" different language versions of the same document, by: i) pinpointing topically related pieces of information in the different languages, ii) identifying information that is missing or less detailed in one of the two versions, iii) translating this in the appropriate language, iv) inserting it in the appropriate place. Progress along such directions will allow users to share more easily content across language boundaries.
This paper presents a question/answering system for searching the web. The system accepts natural... more This paper presents a question/answering system for searching the web. The system accepts natural language questions in Italian and returns as answer a ranked list of document passages together with the URL of the whole document. Three crucial aspects related to the web scenario have been investigated: the linguistic expansion of the query, the optimization of the search boolean expression,
Abstract This paper overviews FBK's participation in the Main and KBP Va... more Abstract This paper overviews FBK's participation in the Main and KBP Validation Pilot task organized within the RTE6 Evaluation Campaign. Our submissions have been produced running the EDITS (Edit Distance Textual Entailment Suite) open source RTE package, ...
This paper presents a general overview of answer-driven reasoning, which may serve as a common fr... more This paper presents a general overview of answer-driven reasoning, which may serve as a common framework for a variety of Information Retrieval related activities. The underlying assumption is that a piece of information in a given document is as relevant as the extent to which such information represents the answer to a possible user's question. The approach relies on the application of answer procedures to a source data (e.g. a text collection) to mine question/answer correlations. The outcome can be exploited for different purposes, ranging from the creation of a Q/A index specifically targeted to intelligent Information Retrieval, Question Answering and Summarization, to the augmentation of existing ontologies in the Semantic Web scenario.
We address a core aspect of the multilingual content synchronization task: the identifica-tion of... more We address a core aspect of the multilingual content synchronization task: the identifica-tion of novel, more informative or semanti-cally equivalent pieces of information in two documents about the same topic. This can be seen as an application-oriented variant of tex-tual entailment recognition where: i) T and H are in different languages, and ii) entail-ment relations between T and H have to be checked in both directions. Using a combi-nation of lexical, syntactic, and semantic fea-tures to train a cross-lingual textual entailment system, we report promising results on differ-ent datasets.
We address two challenges for automatic machine translation evaluation: a) avoiding the use of re... more We address two challenges for automatic machine translation evaluation: a) avoiding the use of reference translations, and b) focusing on adequacy estimation. From an economic perspective, getting rid of costly hand-crafted reference translations (a) permits to alleviate the main bottleneck in MT evaluation. From a system evaluation perspective, pushing semantics into MT (b) is a necessity in order to complement the shallow methods currently used overcoming their limitations. Casting the problem as a cross-lingual textual entailment application, we experiment with different benchmarks and evaluation settings. Our method shows high correlation with human judgements and good results on all datasets without relying on reference translations.
We address two issues related to the development of systems for Recognizing Textual Entailment. T... more We address two issues related to the development of systems for Recognizing Textual Entailment. The first is the impossibility to capitalize on lessons learned over the different datasets available, due to the changing nature of traditional RTE evaluation settings. The second is the lack of simple ways to assess the results achieved by our system on a given training corpus, and figure out its real potential on unseen test data. Our contribution is the extension of an open-source RTE package with an automatic way to explore the large search space of possible configurations, in order to select the most promising one over a given dataset. From the developers' point of view, the efficiency and ease of use of the system, together with the good results achieved on all previous RTE datasets, represent a useful support, providing an immediate term of comparison to position the results of their approach.
This paper overviews FBK's participation in the Cross-Lingual Textual Entailment for Content ... more This paper overviews FBK's participation in the Cross-Lingual Textual Entailment for Content Synchronization task organized within SemEval-2012. Our participation is characterized by using cross-lingual matching features extracted from lexical and semantic phrase tables and dependency relations. The features are used for multi-class and binary classification using SVMs. Using a combination of lexical, syntactic, and semantic features to create a cross-lingual textual entailment system, we report on experiments over the provided dataset. Our best run achieved an accuracy of 50.4% on the Spanish-English dataset (with the average score and the median system respectively achieving 40.7% and 34.6%), demonstrating the effectiveness of a "pure" cross-lingual approach that avoids intermediate translations.
This paper describes the participation of FBK in the Semantic Textual Similarity (STS) task organ... more This paper describes the participation of FBK in the Semantic Textual Similarity (STS) task organized within Semeval 2012. Our approach explores lexical, syntactic and semantic machine translation evaluation metrics combined with distributional and knowledge-based word similarity metrics. Our best model achieves 60.77% correlation with human judgements (Mean score) and ranked 20 out of 88 submitted runs in the Mean ranking, where the average correlation across all the sub-portions of the test set is considered.
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), 2015
The capability of handling anaphora is becoming a key feature for Question Answering systems, as ... more The capability of handling anaphora is becoming a key feature for Question Answering systems, as it can play a crucial role at different stages of the QA loop. At the question processing stage, on which this paper is focused, it enhances the treatment of follow-up questions, allowing for a more natural interaction with the user. As much as the QA task evolves towards a realistic dialogue-based scenario, one of the concrete problems raised by follow-up questions is tracking their actual referent. Each question may in fact refer to the topic of the session, to an answer given to an earlier question, or to a new entity it introduces in the dialogue. Focusing on the referent traceability problem, we present and experiment with a possible data-driven solution which exploits simple features of the input question and its surrounding context (the target of the session, and the previous questions) to inform the next phases of the QA process.
Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02, 2001
ABSTRACT Answer Validation is an emerging topic in Question Answering, where open domain systems ... more ABSTRACT Answer Validation is an emerging topic in Question Answering, where open domain systems are often required to rank huge amounts of candidate answers. We present a novel approach to answer validation based on the intuition that the amount of implicit knowledge which connects an answer to a question can be quantitatively estimated by exploiting the redundancy of Web information. Experiments carried out on the TREC-2001 judged-answer collection show that the approach achieves a high level of performance (i.e. 81% success rate). The simplicity and the efficiency of this approach make it suitable to be used as a module in Question Answering systems.
Proceedings of the Eighth Annual International Symposium on Wikis and Open Collaboration - WikiSym '12, 2012
ABSTRACT CoSyne is a content synchronization system for assisting users and organizations involve... more ABSTRACT CoSyne is a content synchronization system for assisting users and organizations involved in the maintenance of multilingual wikis. The system allows users to explore the diversity of multilingual content using a monolingual view. It provides suggestions for content modification based on additional or more specific information found in other language versions, and enables seamless integration of automatically translated sentences while giving users the flexibility to edit, correct and control eventual changes to the wiki page. To support these tasks, CoSyne employs state-of-the-art machine translation and natural language processing techniques.
Proceedings of the 7th International Symposium on Wikis and Open Collaboration - WikiSym '11, 2011
Wikis allow a large base of contributors easy access to shared content, and freedom in editing it... more Wikis allow a large base of contributors easy access to shared content, and freedom in editing it. One of the side-effects of this freedom was the emergence of parallel and independently evolving versions in a variety of languages, reflecting the multilingual background of the pool of contributors. For the Wiki to properly represent the user-added content, this should be fully available in all its languages. Working on parallel Wikis in several European languages, we investigate the possibility to "synchronize" different language versions of the same document, by: i) pinpointing topically related pieces of information in the different languages, ii) identifying information that is missing or less detailed in one of the two versions, iii) translating this in the appropriate language, iv) inserting it in the appropriate place. Progress along such directions will allow users to share more easily content across language boundaries.
This paper presents a question/answering system for searching the web. The system accepts natural... more This paper presents a question/answering system for searching the web. The system accepts natural language questions in Italian and returns as answer a ranked list of document passages together with the URL of the whole document. Three crucial aspects related to the web scenario have been investigated: the linguistic expansion of the query, the optimization of the search boolean expression,
Uploads
Papers by Matteo Negri