Mining Significant Associations in Large Scale Text Corpora Prabhakar Raghavan and Panayiotis Tsaparas   View Paper (PDF)   Return to Main-Track Regular Papers Abstract Mining large-scale text corpora is an essential step in extracting the key themes in a corpus. We motivate a quanti-tative measure for significant associations through the distributions of pairs and triplets of co-occurring words. We consider the algorithmic problem of efficiently enumerat-ing such significant associations and present pruning algorithms for these problems, with theoretical as well as empirical analyses. Our algorithms make use of two novel mining methods: (1) matrix mining, and (2) shortened documents. We present evidence from a diverse set of documents that our measure does in fact elicit interesting co-occurrences. DiSC'03 © 2003 Association for Computing Machinery