Rutgers interactive track at TREC 2002

Journal of The American Society for Information Science and Technology, 2002

This article compares search effectiveness when using query-based Internet search (via the Google search engine), directory-based search (via Yahoo), and phrasebased query reformulation-assisted search (via the Hyperindex browser) by means of a controlled, user-based experimental study. The focus was to evaluate aspects of the search process. Cognitive load was measured using a secondary digit-monitoring task to quantify the effort of the user in various search states; independent relevance judgements were employed to gauge the quality of the documents accessed during the search process and time was monitored as a function of search state. Results indicated directory-based search does not offer increased relevance over the query-based search (with or without query formulation assistance), and also takes longer. Query reformulation does significantly improve the relevance of the documents through which the user must trawl, particularly when the formulation of query terms is more difficult. However, the improvement in document relevance comes at the cost of increased search time, although this difference is quite small when the search is self-terminated. In addition, the advantage of the query reformulation seems to occur as a consequence of providing more discriminating terms rather than by increasing the length of queries.

Journal of the American Society for …, 2009

The organisation, content and presentation of document surrogates has a substantial impact on the effectiveness of web search result interfaces. Most interfaces include textual information, including for example the document title, URL, and a short query-biased summary of the content. Other interfaces include additional browsing features, such as topic clustering, or thumbnails of the web pages. In this study we analyse three search interfaces, and compare the effectiveness of textual information and additional browsing features. Our analysis indicates that most users spend a substantially larger proportion of time looking at text information, and that those interfaces that focus on text-based representations of document content tend to lead to quicker task completion times for named-page finding search tasks.

2007

2007

Proceedings of the ACM International Conference on Digital Libraries, 2007

In this paper we present the results of a study that investigates the relationships between search tasks, information architecture, and interaction style. Three kinds of search tasks (simple lookup, complex lookup and exploratory) were performed using three different user interfaces (standard web site, hierarchical text-based faceted interface, and dynamic query faceted interface) for a large-scale public corpus containing semi-structured statistical data and reports. Twenty-eight people conducted the three kinds of searches in a between-subjects study and twelve others conducted the three kinds of searches on all three systems in a within-subjects study. Quantitative results demonstrate that the alternative general-purpose user interfaces that accept automated structuring of data offer comparable effectiveness, efficiency, and aesthetics to manually constructed architectures. Qualitative results demonstrate the manual architectures are favored.

2008

2009

Abstract Search result organisation and presentation is an important component of a Web search system, it can have a substantial impact on the ability of users to find useful information. In this study we compare the effectiveness of three publicly available search interfaces for supporting navigational search tasks. The three interfaces vary primarily in the proportion of visual versus textual cues that are used to display a search result.

Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '00, 2000

This article compares search effectiveness when using query-based lnternet search (via the Google search engine), directory-based search (via Yahoo) and phrasebased query reformulation assisted search (via the Hypefindex browser) by means of a controlled, userbased experimental study. The focus was to evaluate aspects of the search process. Cognitive load was • measured using a secondary digit-monitoring task to quantify the effort of the user in various search states; independent relevance judgements were employed to gauge the quality of the documents accessed during the search process. Time was monitored in various search states. Results indicated the directory-based search does not offer increased relevance over the query-based search (with or without query formulation assistance), and also takes longer. Query reformulation does significantly improve the relevance of the documents through which the user must trawl versus standard query-based internet search. However, the improvement in document relevance comes at the cost of increased search time and increased cognitive load. Keywords: navigation versus ad hoc search, monitoring user behaviour to improve search, field/empirical studies of the reformation seeking process, testing methodology. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that capias are not made or dtstributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a foe.

Proceedings of the 2004 joint ACM/IEEE conference on Digital libraries - JCDL '04, 2004