|
|
|
                                          
|
Return to Posters
A new measure, AnchorMap, is introduced to evaluate how close two document retrieval rankings are to each other. It is shown that AnchorMap scores, when run on a set of initial ranked document lists from 8 different systems, are very highly correlated with categorization of topics as easy or hard, and separately, are highly correlated with those topics on which blind feedback works. In another experiment, AnchorMap is used to compare the initial ranked document list from a single system against the ranked document list from that system after blind feedback. Again, high AnchorMap values are highly correlated with both topic difficulty and successful application of blind feedback. Both experiments are examples of using properties of a topic which are independent of relevance information to predict the actual performance of IR systems on the topic. Initial experiments to attempt to improve retrieval performance based upon AnchorMap failed; the causes for failure are discussed.
@inproceedings{1009093, author = {Chris Buckley}, title = {Topic prediction based on comparative retrieval rankings}, booktitle = {SIGIR '04: Proceedings of the 27th annual international conference on Research and development in information retrieval}, year = {2004}, isbn = {1-58113-881-4}, pages = {506--507}, location = {Sheffield, United Kingdom}, doi = {http://doi.acm.org/10.1145/1008992.1009093}, publisher = {ACM Press}, }  ©2005 Association for Computing Machinery |
