Seeking information relevant to a topic of interest has become a common task in our daily activit... more Seeking information relevant to a topic of interest has become a common task in our daily activities. However, searching the Web using current technologies still presents many limitations. One of the main limitations is that existing tools for searching the Web restrict user queries to a small number of terms. As a result, a single query may not reflect the user information needs at a sufficient level of detail. In addition, even if longer queries were allowed, the user may not find the right terms to supply appropriate queries, or may not be willing to put the effort required to explicitly describe his or her information needs. Another limitation of today's search tools is that they are not capable of performing qualitative inference on the suggestions they offer. For certain domains, such as news or scientific articles, a good amount of structural information can be usefully exploited to extract meaningful content. This can help sort out the material returned by a search engine and to perform a qualitative analysis to warrant some of the search results. This paper shows how to enhance current search engines capabilities by (1) taking advantage of the user context, and (2) ranking search results based on preferential criteria provided by the user. We describe ongoing research on the use of context-specific terms to refine Web search and on the use of a defeasible argumentation framework to prioritize search results.
Peer network systems are becoming an increasingly important development in Web search technology.... more Peer network systems are becoming an increasingly important development in Web search technology. To provide insight into the comparison of different peer network systems, proper evaluation criteria are needed. This paper proposes a novel evaluation framework for peer search systems based on the concept of semantic locality. In order for the network to be functional, its dynamic communication topology must match the semantic clustering of peers. We introduce two criteria to evaluate the semantic locality of a peer network. First, the "small-world" topology of the network. Second, we use topical semantic similarity to monitor the quality of a peer's neighbors over time by looking at whether a peer chooses semantically appropriate neighbors to route its queries. Furthermore, we evaluate the quality of search results based on the notions of global coherence and coverage. These two performance functions generalize the well known IR measures of precision and recall by relaxing the condition that all relevant resources are identified in advance. We demonstrate the feasibility of the proposed evaluation framework by presenting several simulation experiments conducted with different types of peer network systems. The results of these simulations suggest that our methodology can be effectively applied to the evaluation of peer based Web search.
This paper proposes an incremental method that can be used by an intelligent system to learn bett... more This paper proposes an incremental method that can be used by an intelligent system to learn better descriptions of a thematic context. The method starts with a small number of terms selected from a simple description of the topic under analysis and uses this description as the initial search context. Using these terms, a set of queries are built and submitted to a search engine. New documents and terms are used to refine the learned vocabulary. Evaluations performed on a large number of topics indicate that the learned vocabulary is much more effective than the original one at the time of constructing queries to retrieve relevant material. ⋆
The absence of reliable and efficient techniques to evaluate information retrieval systems has be... more The absence of reliable and efficient techniques to evaluate information retrieval systems has become a bottleneck in the development of novel retrieval methods. In traditional approaches users or hired evaluators provide manual assessments of relevance. However these approaches are neither efficient nor reliable since they do not scale with the complexity and heterogeneity of available digital information. Automatic approaches, on the other hand, could be efficient but disregard semantic data, which is usually important to assess the actual performance of the evaluated methods. This article proposes to use topic ontologies and semantic similarity data derived from these ontologies to implement an automatic semantic evaluation framework for information retrieval systems. The use of semantic similarity data allows to capture the notion of partial relevance, generalizing traditional evaluation metrics, and giving rise to novel performance measures such as semantic precision and semantic harmonic mean. The validity of the approach is supported by user studies and the application of the proposed framework is illustrated with the evaluation of topical retrieval systems. The evaluated systems include a baseline, a supervised version of the Bo1 query refinement method and two multi-objective evolutionary algorithms for context-based retrieval. Finally, we discuss the advantages of applying evaluation metrics that account for semantic similarity data and partial relevance over existing metrics based on the notion of total relevance.
This chapter proposed a collaborative peer network application called 6Search (6S) to address the... more This chapter proposed a collaborative peer network application called 6Search (6S) to address the scalability limitations of centralized search engines. Each peer crawls the Web in a focused way, guided by its user's information context. Through this approach, better (distributed) coverage can be achieved. Each peer also acts as a search "servent" (server + client) by submitting and responding to queries to/from its neighbors. This search process has no centralized bottleneck. Peers depend on a local adaptive routing algorithm to dynamically change the topology of the peer network and search for the best neighbors to answer their queries. We present and evaluate learning techniques to improve local query routing. We validate prototypes of the 6S network via simulations with 500 70 − model users based on actual Web crawls. We find that the network topology rapidly converges from a random network to a small world network, with clusters emerging from user communities with shared interests. We finally compare the quality of the results with those obtained by centralized search engines such as Google.
Institutional Repositories are collections of digital resources organized to facilitate their lon... more Institutional Repositories are collections of digital resources organized to facilitate their long term access. Two fundamental issues that need to be addressed at the moment of implementing these repositories are their maintenance and search instruments. This article describes an intelligent tool whose goal is to overcome certain general limitations encountered in current mechanisms for managing Institutional Repositories. The novel aspects of the proposed tool are the use of previous examples to generate suggestions for cataloging incoming digital resources and the ability to reflect a thematic context during the search process. The proposed tool applies casebased reasoning in combination with information retrieval techniques to take advantage of a large set of previously classified resources with the purpose of supporting the cataloger's task. The tool has been evaluated using a collection consisting of 9792 training examples from the SciELO electronic library and a test set containing 100 articles coming from different disciplines.
Improvement in information retrieval systems is largely dependent on the ability to evaluate them... more Improvement in information retrieval systems is largely dependent on the ability to evaluate them. In order to assess the effectiveness of a retrieval system, test collections are needed. In traditional approaches users or hired evaluators provide manual assessments of relevance. However this does not scale with the complexity and heterogeneity of available digital information. This paper proposes to use topic ontologies and semantic similarity data to alleviate the efforts needed by human assessors to evaluate the rapidly expanding set of competing information retrieval methods. After providing experimental evidence supporting the validity of our approach we illustrate its application with an example in which the proposed evaluation procedure is used to assess the effectiveness of topical retrieval systems.
A main challenge for Web content classification is how to model the input data. This paper discus... more A main challenge for Web content classification is how to model the input data. This paper discusses the application of two text modeling approaches, Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA), in the Web page classification task. We report results on a comparison of these two approaches using different vocabularies consisting of links and text. Both models are evaluated using different numbers of latent topics. Finally, we evaluate a hybrid latent variable model that combines the latent topics resulting from both LSA and LDA. This new approach turns out to be superior to the basic LSA and LDA models. In our experiments with categories and pages obtained from the ODP web directory the hybrid model achieves an averaged F-measure value of 0.852 and an averaged ROC value of 0.96.
On the Move to Meaningful Internet Systems: …, Jan 1, 2008
Context-based Web search has become an important research area and many strategies have been prop... more Context-based Web search has become an important research area and many strategies have been proposed to reflect contextual information in search queries. Despite the success of some of these proposals they still have serious limitations due to their inability to bridge the terminology gap existing between the user context description and the relevant documents' vocabulary. This paper presents a quantitative technique to learn vocabularies useful for describing the theme of a context under analysis. The enriched vocabulary allows the formulation of search queries to identify resources with higher precision than those identified using the initial vocabulary. Rigorous experimentation leads us to conclude that the proposed technique is superior to a baseline and other well-known query reformulation techniques.
Artificial Intelligence Applications …, Jan 1, 2006
Over the last few years, argumentation has been gaining increasing importance in several Al-relat... more Over the last few years, argumentation has been gaining increasing importance in several Al-related areas, mainly as a vehicle for facilitating rationally justifiable decision making when handling incomplete and potentially inconsistent information. In this setting, user support systems can rely on argumentation techniques to automatize reasoning and decision making in several situations such as the handling of complex policies or managing change in dynamic environments. This paper presents a generic argument-based approach to characterize user support systems, in which knowledge representation and inference are captured in terms of Defeasible Logic Progranmiing, a general-purpose defeasible argumentation formalism based on logic progranmung. We discuss a particular apphcation which has emerged as an instance of this approach oriented towards providing user decision support for web search.
Seeking information relevant to a topic of interest has become a common task in our daily activit... more Seeking information relevant to a topic of interest has become a common task in our daily activities. However, searching the Web using current technologies still presents many limitations. One of the main limitations is that existing tools for searching the Web restrict user queries to a small number of terms. As a result, a single query may not reflect the user information needs at a sufficient level of detail. In addition, even if longer queries were allowed, the user may not find the right terms to supply appropriate queries, or may not be willing to put the effort required to explicitly describe his or her information needs. Another limitation of today's search tools is that they are not capable of performing qualitative inference on the suggestions they offer. For certain domains, such as news or scientific articles, a good amount of structural information can be usefully exploited to extract meaningful content. This can help sort out the material returned by a search engine and to perform a qualitative analysis to warrant some of the search results. This paper shows how to enhance current search engines capabilities by (1) taking advantage of the user context, and (2) ranking search results based on preferential criteria provided by the user. We describe ongoing research on the use of context-specific terms to refine Web search and on the use of a defeasible argumentation framework to prioritize search results.
Peer network systems are becoming an increasingly important development in Web search technology.... more Peer network systems are becoming an increasingly important development in Web search technology. To provide insight into the comparison of different peer network systems, proper evaluation criteria are needed. This paper proposes a novel evaluation framework for peer search systems based on the concept of semantic locality. In order for the network to be functional, its dynamic communication topology must match the semantic clustering of peers. We introduce two criteria to evaluate the semantic locality of a peer network. First, the "small-world" topology of the network. Second, we use topical semantic similarity to monitor the quality of a peer's neighbors over time by looking at whether a peer chooses semantically appropriate neighbors to route its queries. Furthermore, we evaluate the quality of search results based on the notions of global coherence and coverage. These two performance functions generalize the well known IR measures of precision and recall by relaxing the condition that all relevant resources are identified in advance. We demonstrate the feasibility of the proposed evaluation framework by presenting several simulation experiments conducted with different types of peer network systems. The results of these simulations suggest that our methodology can be effectively applied to the evaluation of peer based Web search.
This paper proposes an incremental method that can be used by an intelligent system to learn bett... more This paper proposes an incremental method that can be used by an intelligent system to learn better descriptions of a thematic context. The method starts with a small number of terms selected from a simple description of the topic under analysis and uses this description as the initial search context. Using these terms, a set of queries are built and submitted to a search engine. New documents and terms are used to refine the learned vocabulary. Evaluations performed on a large number of topics indicate that the learned vocabulary is much more effective than the original one at the time of constructing queries to retrieve relevant material. ⋆
The absence of reliable and efficient techniques to evaluate information retrieval systems has be... more The absence of reliable and efficient techniques to evaluate information retrieval systems has become a bottleneck in the development of novel retrieval methods. In traditional approaches users or hired evaluators provide manual assessments of relevance. However these approaches are neither efficient nor reliable since they do not scale with the complexity and heterogeneity of available digital information. Automatic approaches, on the other hand, could be efficient but disregard semantic data, which is usually important to assess the actual performance of the evaluated methods. This article proposes to use topic ontologies and semantic similarity data derived from these ontologies to implement an automatic semantic evaluation framework for information retrieval systems. The use of semantic similarity data allows to capture the notion of partial relevance, generalizing traditional evaluation metrics, and giving rise to novel performance measures such as semantic precision and semantic harmonic mean. The validity of the approach is supported by user studies and the application of the proposed framework is illustrated with the evaluation of topical retrieval systems. The evaluated systems include a baseline, a supervised version of the Bo1 query refinement method and two multi-objective evolutionary algorithms for context-based retrieval. Finally, we discuss the advantages of applying evaluation metrics that account for semantic similarity data and partial relevance over existing metrics based on the notion of total relevance.
This chapter proposed a collaborative peer network application called 6Search (6S) to address the... more This chapter proposed a collaborative peer network application called 6Search (6S) to address the scalability limitations of centralized search engines. Each peer crawls the Web in a focused way, guided by its user's information context. Through this approach, better (distributed) coverage can be achieved. Each peer also acts as a search "servent" (server + client) by submitting and responding to queries to/from its neighbors. This search process has no centralized bottleneck. Peers depend on a local adaptive routing algorithm to dynamically change the topology of the peer network and search for the best neighbors to answer their queries. We present and evaluate learning techniques to improve local query routing. We validate prototypes of the 6S network via simulations with 500 70 − model users based on actual Web crawls. We find that the network topology rapidly converges from a random network to a small world network, with clusters emerging from user communities with shared interests. We finally compare the quality of the results with those obtained by centralized search engines such as Google.
Institutional Repositories are collections of digital resources organized to facilitate their lon... more Institutional Repositories are collections of digital resources organized to facilitate their long term access. Two fundamental issues that need to be addressed at the moment of implementing these repositories are their maintenance and search instruments. This article describes an intelligent tool whose goal is to overcome certain general limitations encountered in current mechanisms for managing Institutional Repositories. The novel aspects of the proposed tool are the use of previous examples to generate suggestions for cataloging incoming digital resources and the ability to reflect a thematic context during the search process. The proposed tool applies casebased reasoning in combination with information retrieval techniques to take advantage of a large set of previously classified resources with the purpose of supporting the cataloger's task. The tool has been evaluated using a collection consisting of 9792 training examples from the SciELO electronic library and a test set containing 100 articles coming from different disciplines.
Improvement in information retrieval systems is largely dependent on the ability to evaluate them... more Improvement in information retrieval systems is largely dependent on the ability to evaluate them. In order to assess the effectiveness of a retrieval system, test collections are needed. In traditional approaches users or hired evaluators provide manual assessments of relevance. However this does not scale with the complexity and heterogeneity of available digital information. This paper proposes to use topic ontologies and semantic similarity data to alleviate the efforts needed by human assessors to evaluate the rapidly expanding set of competing information retrieval methods. After providing experimental evidence supporting the validity of our approach we illustrate its application with an example in which the proposed evaluation procedure is used to assess the effectiveness of topical retrieval systems.
A main challenge for Web content classification is how to model the input data. This paper discus... more A main challenge for Web content classification is how to model the input data. This paper discusses the application of two text modeling approaches, Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA), in the Web page classification task. We report results on a comparison of these two approaches using different vocabularies consisting of links and text. Both models are evaluated using different numbers of latent topics. Finally, we evaluate a hybrid latent variable model that combines the latent topics resulting from both LSA and LDA. This new approach turns out to be superior to the basic LSA and LDA models. In our experiments with categories and pages obtained from the ODP web directory the hybrid model achieves an averaged F-measure value of 0.852 and an averaged ROC value of 0.96.
On the Move to Meaningful Internet Systems: …, Jan 1, 2008
Context-based Web search has become an important research area and many strategies have been prop... more Context-based Web search has become an important research area and many strategies have been proposed to reflect contextual information in search queries. Despite the success of some of these proposals they still have serious limitations due to their inability to bridge the terminology gap existing between the user context description and the relevant documents' vocabulary. This paper presents a quantitative technique to learn vocabularies useful for describing the theme of a context under analysis. The enriched vocabulary allows the formulation of search queries to identify resources with higher precision than those identified using the initial vocabulary. Rigorous experimentation leads us to conclude that the proposed technique is superior to a baseline and other well-known query reformulation techniques.
Artificial Intelligence Applications …, Jan 1, 2006
Over the last few years, argumentation has been gaining increasing importance in several Al-relat... more Over the last few years, argumentation has been gaining increasing importance in several Al-related areas, mainly as a vehicle for facilitating rationally justifiable decision making when handling incomplete and potentially inconsistent information. In this setting, user support systems can rely on argumentation techniques to automatize reasoning and decision making in several situations such as the handling of complex policies or managing change in dynamic environments. This paper presents a generic argument-based approach to characterize user support systems, in which knowledge representation and inference are captured in terms of Defeasible Logic Progranmiing, a general-purpose defeasible argumentation formalism based on logic progranmung. We discuss a particular apphcation which has emerged as an instance of this approach oriented towards providing user decision support for web search.
Uploads
Papers by Ana Maguitman