|
|
|
     |
                                         |
Return to Session: Theory 1
We argue that the ability to identify pairs of related terms is at the heart of what makes spectral retrieval work in practice. Schemes such as latent semantic indexing (LSI) and its descendants have this ability in the sense that they can be viewed as computing a matrix of term-term relatedness scores which is then used to expand the given documents (not the queries). For almost all existing spectral retrieval schemes, this matrix of relatedness scores depends on a fixed low-dimensional subspace of the original term space. We instead vary the dimension and study for each term pair the resultin curve of relatedness scores. We find that it is actually the shape of this curve which is indicative for the term-pair relatedness, and not any of the individual relatedness scores on the curve. We derive two simple, parameterless algorithms that detect this shape and that consistently outperform previous methods on a number of test collections. Our curves also shed light on the effectiveness of three fundamental types of variations of the basic LSI scheme.
@inproceedings{1076040, author = {Holger Bast and Debapriyo Majumdar}, title = {Why spectral retrieval works}, booktitle = {SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval}, year = {2005}, isbn = {1-59593-034-5}, pages = {11--18}, location = {Salvador, Brazil}, doi = {http://doi.acm.org/10.1145/1076034.1076040}, publisher = {ACM Press}, address = {New York, NY, USA}, }  ©2006 Association for Computing Machinery |
