|
|
|
                                          
|
Return to Session VIIb: Efficiency and scaling
Web Search Engines provide a large-scale text document retrieval service by processing huge Inverted File indexes. Inverted File indexes allow fast query resolution and good memory utilization since their d-gaps representation can be effectively and efficiently compressed by using variable length encoding methods. This paper proposes and evaluates some algorithms aimed to find an assignment of the document identifiers which minimizes the average values of d-gaps, thus enhancing the effectiveness of traditional compression methods. We ran several tests over the Google contest collection in order to validate the techniques proposed. The experiments demonstrated the scalability and effectiveness of our algorithms. Using the proposed algorithms, we were able to sensibly improve (up to 20.81%) the compression ratios of several encoding schemes.
@inproceedings{1009046, author = {Fabrizio Silvestri and Salvatore Orlando and Raffaele Perego}, title = {Assigning identifiers to documents to enhance the clustering property of fulltext indexes}, booktitle = {SIGIR '04: Proceedings of the 27th annual international conference on Research and development in information retrieval}, year = {2004}, isbn = {1-58113-881-4}, pages = {305--312}, location = {Sheffield, United Kingdom}, doi = {http://doi.acm.org/10.1145/1008992.1009046}, publisher = {ACM Press}, }  ©2005 Association for Computing Machinery |
