Skip to main content

Ngoc Anh

Followers

0

Following

1

Public Views

José Nureña Angulo

Queensland University of Technology

Marijana Pavunc

Mamikon Ginovyan

Sergey Lototsky

Hossein Haghbin

London School of Economics and Political Science

Interests

Uploads

Papers by Ngoc Anh

Markovian representation and prediction of stochastic processes with time-varying rational spectrum

Signal Processing, 1991

This paper solves the spectral factorization problem for a class of processes with time-dependent... more This paper solves the spectral factorization problem for a class of processes with time-dependent rational spectral density. As a consequence, the prediction problem for these processes is solved based on the integral operator method. In addition, the connection between processes with time-dependent rational spectral density and time-dependent ARMA processes is discussed. Zusammenfassung. In diesem Beitrag wird das spektrale Faktorisierungsproblem fiir eine Klasse stochastischer Prozesse mit zeitabh~ingigem, rationalem Leistungsspektrum gel6st. Als eine Folgerung aus diesem Resultat wird das Priidikationsproblem fiir diese Prozesse nach der lntegraloperator-Methode gel6st. Zus~itzlich wird der Zusammenhang zwischen Prozessen mit zeitabhiingigem, rationalem Leistungsspektrum und zeitabh~ingigen ARMA-Prozessen diskutiert. R~sum~. Le manuscrit r6soud le probl~me de la factorisation spectrale pour une classe de processus ayant pour densit6 spectrale une fraction rationnelle d6pendant du temps. Comme consequence, le probl~me de pr6diction de ces processus est r6solu en utilisant la m~thode de l'op~rateur integral. De plus, le lien entre des processus ayant pour densit~ spectrale une fraction rationnelle d6pendant du temps et des processus ARMA d~pendant du temps est discut~.

Improved Retrieval Effectiveness Through Impact Transformation

Users of web search engines are notoriously parsimonious in their use of search terms, and search... more Users of web search engines are notoriously parsimonious in their use of search terms, and search effectiveness has tended to be relatively poor on the resulting short queries, especially when compared against the good performance attained by recent systems when working with long TREC-like queries. In this paper we examine the reasons why short queries are hard to deal with, and propose a modification to the vector-space paradigm, which we call impact transformation. Experiments with a collection of web data and web queries shows that impact transformation significantly boosts retrieval effectiveness. Moreover, the use of quantised values in the transformation allows extremely fast query processing.

Robust and Web Retrieval with Document-Centric Integral Impacts

Text REtrieval Conference, 2003

Simplified similarity scoring using term ranks

We propose a method for document ranking that combines a simple document-centric view of text, an... more We propose a method for document ranking that combines a simple document-centric view of text, and fast evaluation strategies that have been developed in connection with the vector space model. The new method defines the importance of a term within a document qualitatively rather than quantitatively, and in doing so reduces the need for tuning parameters. In addition, the method supports very fast query processing, with most of the computation carried out on small integers, and dynamic pruning an effective option. Experiments on a wide range of TREC data show that the new method provides retrieval effectiveness as good as or better than the Okapi BM25 formulation, and variants of language models.

Pruned query evaluation using pre-computed impacts

Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '06, 2006

Exhaustive evaluation of ranked queries can be expensive, particularly when only a small subset o... more Exhaustive evaluation of ranked queries can be expensive, particularly when only a small subset of the overall ranking is required, or when queries contain common terms. This concern gives rise to techniques for dynamic query pruning, that is, methods for eliminating redundant parts of the usual exhaustive evaluation, yet still generating a demonstrably "good enough" set of answers to the query. In this work we propose new pruning methods that make use of impact-sorted indexes. Compared to exhaustive evaluation, the new methods reduce the amount of computation performed, reduce the amount of memory required for accumulators, reduce the amount of data transferred from disk, and at the same time allow performance guarantees in terms of precision and mean average precision. These strong claims are backed by experiments using the TREC Terabyte collection and queries.

Vector-space ranking with effective early termination

Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '01, 2001

Considerable research effort has been invested in improving the effectiveness of information retr... more Considerable research effort has been invested in improving the effectiveness of information retrieval systems. Techniques such as relevance feedback, thesaural expansion, and pivoting all provide better quality responses to queries when tested in standard evaluation frameworks. But such enhancements can add to the cost of evaluating queries. In this paper we consider the pragmatic issue of how to improve the cost-effectiveness of searching. We describe a new inverted file structure using quantized weights that provides superior retrieval effectiveness compared to conventional inverted file structures when early termination heuristics are employed. That is, we are able to reach similar effectiveness levels with less computational cost, and so provide a better cost/performance compromise than previous inverted file organisations.

Impact transformation

Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '02, 2002

We extend the applicability of impact transformation, which is a technique for adjusting the term... more We extend the applicability of impact transformation, which is a technique for adjusting the term weights assigned to documents so as to boost the effectiveness of retrieval when short queries are applied to large document collections. In conjunction with techniques called quantization and thresholding, impact transformation allows improved query execution rates compared to traditional vector-space similarity computations, as the number of arithmetic operations can be reduced. The transformation also facilitates a new dynamic query pruning heuristic. We give results based upon the TREC web data that show the combination of these various techniques to yield highly competitive retrieval, in terms of both effectiveness and efficiency, for both short and long queries.

Structured Index Organizations for High-Throughput Text Querying

Lecture Notes in Computer Science, 2006

Inverted indexes are the preferred mechanism for supporting contentbased queries in text retrieva... more Inverted indexes are the preferred mechanism for supporting contentbased queries in text retrieval systems, with the various data items usually stored compressed in some way. But different query modalities require that different information be held in the index. For example, phrase querying requires that word offsets be held as well as document numbers. In this study we describe an inverted index organization that provides efficient support for all of conjunctive Boolean queries, ranked queries, and phrase queries. Experimental results on a 426 GB document collection show that the methods we describe provide fast evaluation of all three querying modes.

Pruning strategies for mixed-mode querying

Proceedings of the 15th ACM international conference on Information and knowledge management - CIKM '06, 2006

Abstract Web information retrieval systems face a range of unique challenges, not the least of wh... more

Binary Codes for Non-Uniform Sources

Data Compression Conference

Index compression using 64-bit words

Software: Practice and Experience, 2010

Modern computers typically make use of 64-bit words as the fundamental unit of data access. Howev... more Modern computers typically make use of 64-bit words as the fundamental unit of data access. However the decade-long migration from 32-bit architectures has not been reflected in compression technology, because of a widespread assumption that effective compression techniques operate in terms of bits or bytes, rather than words. Here we demonstrate that the use of 64-bit access units, especially in connection with word-bounded codes, does indeed provide the opportunity for improving the compression performance. In particular, we extend several 32-bit word-bounded coding schemes to 64-bit operation and explore their uses in information retrieval applications. Our results show that the Simple-8b approach, a 64-bit word-bounded code, is an excellent self-skipping code, and has a clear advantage over its competitors in supporting fast query evaluation when the data being compressed represents the inverted index for a large text collection. The advantages of the new code also accrue on 32-bit architectures, and for all of Boolean, ranked, and phrase queries; which means that it can be used in any situation.

Inverted Index Compression Using Word-Aligned Binary Codes

Information Retrieval, 2005

We examine index representation techniques for document-based inverted files, and present a mecha... more We examine index representation techniques for document-based inverted files, and present a mechanism for compressing them using word-aligned binary codes. The new approach allows extremely fast decoding of inverted lists during query processing, while providing compression rates better than other high-throughput representations. Results are given for several large text collections in support of these claims, both for compression effectiveness and query efficiency.

Binary codes for locally homogeneous sequences

Information Processing Letters, 2006

Static codes for the non-negative integers have a range of applications, including in the storage... more Static codes for the non-negative integers have a range of applications, including in the storage of inverted indexes, and in dictionary-based compression systems. We present two simple codes based on binary representations that are suited for message sequences in which the values are locally homogeneous, but not globally consistent. Experimental results based on typical sequences are given to show the efficacy of the new methods.

Improved word-aligned binary compression for text indexing

IEEE Transactions on Knowledge and Data Engineering, 2006

Index TermsData compaction and compression, textual databases, indexing methods, file organizati... more

Impact-based document retrieval

Page 1. Impact-Based Document Retrieval VO NGOC ANH Department of Computer Science and Software E... more

Passage retrieval with vector space and query-level aspect models

This report describes the joint work by Kyoto University and the University of Melbourne for the ... more This report describes the joint work by Kyoto University and the University of Melbourne for the TREC Genomics Track in 2007. As with 2006, the task for this year was the retrieval of passages from a biomedical document collection. The overall framework of our system from 2006 remains unchanged and is comprised of two parts: a paragraph-level retrieval system and a passage extraction system. These two systems are based on the vector space model and a probabilistic word-based aspect model, respectively. This year, we have adopted numerous changes to our 2007 system which we believe corrected some problems.

Document Ranking for Effectiveness-Efficiency Tradeoffs

Aim: A large-scale document ranking system should be both effective and efficient. Here we summar... more Aim: A large-scale document ranking system should be both effective and efficient. Here we summarize the main features of a prototype system we built for that purpose. The success of the system is demonstrated through its relative performance for the efficiency task of the TREC 2 005 Terabyte Track. Method: Building a document ranking system involves two key decisions: choosing a retrieval model, and choosing a suitable index representation. The former determines the effectiveness of the system, the latter the efficiency; and each of them affects the other. The impact-based document ranking mechanism described by Anh and Moffat [2] was chosen for our system because of its balance between effectiveness and efficiency . In terms of effectiveness, it is highly competitive, although still inferior to advanced language modelling implementations. On the other hand, in terms of efficiency the mechanism is excellent, as it ranks documents using a small number of calculations, all on integer numbers. To further facilitate query efficiency, we compress the inde x using the slide-8 coding scheme [1], which allows an excellent balance between compression ratio and decoding speed.

The Role of Anchor Text in ClueWeb09 Retrieval

Melbourne University at the 2006 Terabyte Track

This report describes the work done at The University of Melbourne for the TREC-2006 Terabyte Tra... more This report describes the work done at The University of Melbourne for the TREC-2006 Terabyte Track. For this track, we participated in all three main tasks. We continued our work with impact-based ranking and sought to reduce indexing as well as query time. However, to support the named-page task, more conventional retrieval mechanisms were also employed. The results show that, in general, the efficiency performance is slightly better than the previous year. The effectiveness level remains the same.

Melbourne University 2005: Enterprise and Terabyte Tracks

This report describes the work done at The University of Melbourne for the TREC-2005 Enterprise a... more This report describes the work done at The University of Melbourne for the TREC-2005 Enterprise and Terabyte Tracks. In the Enterprise Track, we participated in the Discussion Task. We tried a number of different methods to make use of special features of mailing lists to improve retrieval effectiveness, and found the use of thread context to be promising. In the Terabyte Track, we continued our work with impact-based ranking and sought to reduce indexing as well as query time. A new retrieval system has been developed for this purpose and has shown several improvements over our system from last year.

Markovian representation and prediction of stochastic processes with time-varying rational spectrum

Signal Processing, 1991

This paper solves the spectral factorization problem for a class of processes with time-dependent... more This paper solves the spectral factorization problem for a class of processes with time-dependent rational spectral density. As a consequence, the prediction problem for these processes is solved based on the integral operator method. In addition, the connection between processes with time-dependent rational spectral density and time-dependent ARMA processes is discussed. Zusammenfassung. In diesem Beitrag wird das spektrale Faktorisierungsproblem fiir eine Klasse stochastischer Prozesse mit zeitabh~ingigem, rationalem Leistungsspektrum gel6st. Als eine Folgerung aus diesem Resultat wird das Priidikationsproblem fiir diese Prozesse nach der lntegraloperator-Methode gel6st. Zus~itzlich wird der Zusammenhang zwischen Prozessen mit zeitabhiingigem, rationalem Leistungsspektrum und zeitabh~ingigen ARMA-Prozessen diskutiert. R~sum~. Le manuscrit r6soud le probl~me de la factorisation spectrale pour une classe de processus ayant pour densit6 spectrale une fraction rationnelle d6pendant du temps. Comme consequence, le probl~me de pr6diction de ces processus est r6solu en utilisant la m~thode de l'op~rateur integral. De plus, le lien entre des processus ayant pour densit~ spectrale une fraction rationnelle d6pendant du temps et des processus ARMA d~pendant du temps est discut~.

Improved Retrieval Effectiveness Through Impact Transformation

Users of web search engines are notoriously parsimonious in their use of search terms, and search... more Users of web search engines are notoriously parsimonious in their use of search terms, and search effectiveness has tended to be relatively poor on the resulting short queries, especially when compared against the good performance attained by recent systems when working with long TREC-like queries. In this paper we examine the reasons why short queries are hard to deal with, and propose a modification to the vector-space paradigm, which we call impact transformation. Experiments with a collection of web data and web queries shows that impact transformation significantly boosts retrieval effectiveness. Moreover, the use of quantised values in the transformation allows extremely fast query processing.

Robust and Web Retrieval with Document-Centric Integral Impacts

Text REtrieval Conference, 2003

Simplified similarity scoring using term ranks

We propose a method for document ranking that combines a simple document-centric view of text, an... more We propose a method for document ranking that combines a simple document-centric view of text, and fast evaluation strategies that have been developed in connection with the vector space model. The new method defines the importance of a term within a document qualitatively rather than quantitatively, and in doing so reduces the need for tuning parameters. In addition, the method supports very fast query processing, with most of the computation carried out on small integers, and dynamic pruning an effective option. Experiments on a wide range of TREC data show that the new method provides retrieval effectiveness as good as or better than the Okapi BM25 formulation, and variants of language models.

Pruned query evaluation using pre-computed impacts

Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '06, 2006

Exhaustive evaluation of ranked queries can be expensive, particularly when only a small subset o... more Exhaustive evaluation of ranked queries can be expensive, particularly when only a small subset of the overall ranking is required, or when queries contain common terms. This concern gives rise to techniques for dynamic query pruning, that is, methods for eliminating redundant parts of the usual exhaustive evaluation, yet still generating a demonstrably "good enough" set of answers to the query. In this work we propose new pruning methods that make use of impact-sorted indexes. Compared to exhaustive evaluation, the new methods reduce the amount of computation performed, reduce the amount of memory required for accumulators, reduce the amount of data transferred from disk, and at the same time allow performance guarantees in terms of precision and mean average precision. These strong claims are backed by experiments using the TREC Terabyte collection and queries.

Vector-space ranking with effective early termination

Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '01, 2001

Considerable research effort has been invested in improving the effectiveness of information retr... more Considerable research effort has been invested in improving the effectiveness of information retrieval systems. Techniques such as relevance feedback, thesaural expansion, and pivoting all provide better quality responses to queries when tested in standard evaluation frameworks. But such enhancements can add to the cost of evaluating queries. In this paper we consider the pragmatic issue of how to improve the cost-effectiveness of searching. We describe a new inverted file structure using quantized weights that provides superior retrieval effectiveness compared to conventional inverted file structures when early termination heuristics are employed. That is, we are able to reach similar effectiveness levels with less computational cost, and so provide a better cost/performance compromise than previous inverted file organisations.

Impact transformation

Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '02, 2002

We extend the applicability of impact transformation, which is a technique for adjusting the term... more We extend the applicability of impact transformation, which is a technique for adjusting the term weights assigned to documents so as to boost the effectiveness of retrieval when short queries are applied to large document collections. In conjunction with techniques called quantization and thresholding, impact transformation allows improved query execution rates compared to traditional vector-space similarity computations, as the number of arithmetic operations can be reduced. The transformation also facilitates a new dynamic query pruning heuristic. We give results based upon the TREC web data that show the combination of these various techniques to yield highly competitive retrieval, in terms of both effectiveness and efficiency, for both short and long queries.

Structured Index Organizations for High-Throughput Text Querying

Lecture Notes in Computer Science, 2006

Inverted indexes are the preferred mechanism for supporting contentbased queries in text retrieva... more Inverted indexes are the preferred mechanism for supporting contentbased queries in text retrieval systems, with the various data items usually stored compressed in some way. But different query modalities require that different information be held in the index. For example, phrase querying requires that word offsets be held as well as document numbers. In this study we describe an inverted index organization that provides efficient support for all of conjunctive Boolean queries, ranked queries, and phrase queries. Experimental results on a 426 GB document collection show that the methods we describe provide fast evaluation of all three querying modes.

Pruning strategies for mixed-mode querying

Proceedings of the 15th ACM international conference on Information and knowledge management - CIKM '06, 2006

Abstract Web information retrieval systems face a range of unique challenges, not the least of wh... more

Binary Codes for Non-Uniform Sources

Data Compression Conference

Index compression using 64-bit words

Software: Practice and Experience, 2010

Modern computers typically make use of 64-bit words as the fundamental unit of data access. Howev... more Modern computers typically make use of 64-bit words as the fundamental unit of data access. However the decade-long migration from 32-bit architectures has not been reflected in compression technology, because of a widespread assumption that effective compression techniques operate in terms of bits or bytes, rather than words. Here we demonstrate that the use of 64-bit access units, especially in connection with word-bounded codes, does indeed provide the opportunity for improving the compression performance. In particular, we extend several 32-bit word-bounded coding schemes to 64-bit operation and explore their uses in information retrieval applications. Our results show that the Simple-8b approach, a 64-bit word-bounded code, is an excellent self-skipping code, and has a clear advantage over its competitors in supporting fast query evaluation when the data being compressed represents the inverted index for a large text collection. The advantages of the new code also accrue on 32-bit architectures, and for all of Boolean, ranked, and phrase queries; which means that it can be used in any situation.

Inverted Index Compression Using Word-Aligned Binary Codes

Information Retrieval, 2005

We examine index representation techniques for document-based inverted files, and present a mecha... more We examine index representation techniques for document-based inverted files, and present a mechanism for compressing them using word-aligned binary codes. The new approach allows extremely fast decoding of inverted lists during query processing, while providing compression rates better than other high-throughput representations. Results are given for several large text collections in support of these claims, both for compression effectiveness and query efficiency.

Binary codes for locally homogeneous sequences

Information Processing Letters, 2006

Static codes for the non-negative integers have a range of applications, including in the storage... more Static codes for the non-negative integers have a range of applications, including in the storage of inverted indexes, and in dictionary-based compression systems. We present two simple codes based on binary representations that are suited for message sequences in which the values are locally homogeneous, but not globally consistent. Experimental results based on typical sequences are given to show the efficacy of the new methods.

Improved word-aligned binary compression for text indexing

IEEE Transactions on Knowledge and Data Engineering, 2006

Index TermsData compaction and compression, textual databases, indexing methods, file organizati... more

Impact-based document retrieval

Page 1. Impact-Based Document Retrieval VO NGOC ANH Department of Computer Science and Software E... more

Passage retrieval with vector space and query-level aspect models

This report describes the joint work by Kyoto University and the University of Melbourne for the ... more This report describes the joint work by Kyoto University and the University of Melbourne for the TREC Genomics Track in 2007. As with 2006, the task for this year was the retrieval of passages from a biomedical document collection. The overall framework of our system from 2006 remains unchanged and is comprised of two parts: a paragraph-level retrieval system and a passage extraction system. These two systems are based on the vector space model and a probabilistic word-based aspect model, respectively. This year, we have adopted numerous changes to our 2007 system which we believe corrected some problems.

Document Ranking for Effectiveness-Efficiency Tradeoffs

Aim: A large-scale document ranking system should be both effective and efficient. Here we summar... more Aim: A large-scale document ranking system should be both effective and efficient. Here we summarize the main features of a prototype system we built for that purpose. The success of the system is demonstrated through its relative performance for the efficiency task of the TREC 2 005 Terabyte Track. Method: Building a document ranking system involves two key decisions: choosing a retrieval model, and choosing a suitable index representation. The former determines the effectiveness of the system, the latter the efficiency; and each of them affects the other. The impact-based document ranking mechanism described by Anh and Moffat [2] was chosen for our system because of its balance between effectiveness and efficiency . In terms of effectiveness, it is highly competitive, although still inferior to advanced language modelling implementations. On the other hand, in terms of efficiency the mechanism is excellent, as it ranks documents using a small number of calculations, all on integer numbers. To further facilitate query efficiency, we compress the inde x using the slide-8 coding scheme [1], which allows an excellent balance between compression ratio and decoding speed.

The Role of Anchor Text in ClueWeb09 Retrieval

Melbourne University at the 2006 Terabyte Track

This report describes the work done at The University of Melbourne for the TREC-2006 Terabyte Tra... more This report describes the work done at The University of Melbourne for the TREC-2006 Terabyte Track. For this track, we participated in all three main tasks. We continued our work with impact-based ranking and sought to reduce indexing as well as query time. However, to support the named-page task, more conventional retrieval mechanisms were also employed. The results show that, in general, the efficiency performance is slightly better than the previous year. The effectiveness level remains the same.

Melbourne University 2005: Enterprise and Terabyte Tracks

This report describes the work done at The University of Melbourne for the TREC-2005 Enterprise a... more This report describes the work done at The University of Melbourne for the TREC-2005 Enterprise and Terabyte Tracks. In the Enterprise Track, we participated in the Discussion Task. We tried a number of different methods to make use of special features of mailing lists to improve retrieval effectiveness, and found the use of thread context to be promising. In the Terabyte Track, we continued our work with impact-based ranking and sought to reduce indexing as well as query time. A new retrieval system has been developed for this purpose and has shown several improvements over our system from last year.