Integrating Mining with Relational Database Systems: Alternatives and Implications
Sunita Sarawagi, Shiby Thomas, Rakesh Agrawal
Full Paper (PDF)

Abstract
Data mining on large data warehouses is becoming increasingly important. In support of this trend, we consider a spectrum of architectural alternatives for coupling mining with database systems. These alternatives include: loose-coupling through a SQL cursor interface; encapsulation of a mining algorithm in a stored procedure; caching the data to a file system on-the-fly and mining; tight-coupling using primarily user-defined functions; and SQL implementations for processing in the DBMS. We comprehensively study the option of expressing the mining algorithm in the form of SQL queries using Association rule mining as a case in point. We consider four options in SQL-92 and six options in SQL enhanced with object-relational extensions (SQL-OR). Our evaluation of the different architectural alternatives shows that from a performance perspective, the Cache-Mine option is superior, although the performance of the SQL-OR option is within a factor of two. Both the Cache-Mine and the SQL-OR approaches incur a higher storage penalty than the loose-coupling approach which performance-wise is a factor of 3 to 4 worse than Cache-Mine. The SQL-92 implementations were too slow to qualify as a competitive option. We also compare these alternatives on the basis of qualitative factors like automatic parallelization, development ease, portability and inter-operability.

References

References, where available, link to the DBLP on the World Wide Web.

[1]
Rakesh Agrawal, Manish Mehta, John C. Shafer, Ramakrishnan Srikant, Andreas Arning, Toni Bollinger: The Quest Data Mining System. KDD 1996: 244-249
[2]
Rakesh Agrawal, Tomasz Imielinski, Arun N. Swami: Mining Association Rules between Sets of Items in Large Databases. SIGMOD Conference 1993: 207-216
[3]
...
[4]
Rakesh Agrawal, John C. Shafer: Parallel Mining of Association Rules. TKDE 8(6): 962-969(1996)
[5]
Rakesh Agrawal, Kyuseok Shim: Developing Tightly-Coupled Data Mining Applications on a Relational Database System. KDD 1996: 287-290
[6]
Sergey Brin, Rajeev Motwani, Jeffrey D. Ullman, Shalom Tsur: Dynamic Itemset Counting and Implication Rules for Market Basket Data. SIGMOD Conference 1997: 255-264
[7]
Donald D. Chamberlin: Using the New DB2: IBM's Object-Relational Database System. Morgan Kaufmann 1996, ISBN 1-55860-373-5
[8]
Usama M. Fayyad, Gregory Piatetsky-Shapiro, Padhraic Smyth, Ramasamy Uthurusamy (Eds.): Advances in Knowledge Discovery and Data Mining. AAAI/MIT Press 1996, ISBN 0-262-56097-6
Contents
[9]
...
[10]
Maurice A. W. Houtsma, Arun N. Swami: Set-Oriented Mining for Association Rules in Relational Databases. ICDE 1995: 25-33
[11]
...
[12]
Tomasz Imielinski, Heikki Mannila: A Database Perspective on Knowledge Discovery. CACM 39(11): 58-64(1996)
[13]
...
[14]
...
[15]
Krishna G. Kulkarni: Object-Oriented Extensions in SQL3: A Status Report. SIGMOD Conference 1994: 478
[16]
Jim Melton, Alan R. Simon: Understanding the New SQL: A Complete Guide. Morgan Kaufmann 1993, ISBN 1-55860-245-3
Contents
[17]
Rosa Meo, Giuseppe Psaila, Stefano Ceri: A New SQL-like Operator for Mining Association Rules. VLDB 1996: 122-133
[18]
...
[19]
Berthold Reinwald, Hamid Pirahesh: SQL Open Heterogeneous Data Access. SIGMOD Conference 1998: 506-507
[20]
...
[21]
...
[22]
Ramakrishnan Srikant, Rakesh Agrawal: Mining Generalized Association Rules. VLDB 1995: 407-419
[23]
Ramakrishnan Srikant, Rakesh Agrawal: Mining Sequential Patterns: Generalizations and Performance Improvements. EDBT 1996: 3-17
[24]
Hannu Toivonen: Sampling Large Databases for Association Rules. VLDB 1996: 134-145
[25]
Dick Tsur, Jeffrey D. Ullman, Serge Abiteboul, Chris Clifton, Rajeev Motwani, Svetlozar Nestorov, Arnon Rosenthal: Query Flocks: A Generalization of Association-Rule Mining. SIGMOD Conference 1998: 1-12
[26]
... Referenced By:
  1. Surajit Chaudhuri: Data Mining and Database Systems: Where is the Intersection? Data Engineering Bulletin 21(1): 4-8(1998)
BIBTEX

@inproceedings{DBLP:conf/sigmod/SarawagiTA98,
author = {Sunita Sarawagi and
Shiby Thomas and
Rakesh Agrawal},
editor = {Laura M. Haas and
Ashutosh Tiwary},
title = {Integrating Mining with Relational Database Systems: Alternatives
and Implications},
booktitle = {SIGMOD 1998, Proceedings ACM SIGMOD International Conference
on Management of Data, June 2-4, 1998, Seattle, Washington, USA},
publisher = {ACM Press},
year = {1998},
isbn = {0-89791-955-5},
pages = {343-354},
crossref = {DBLP:conf/sigmod/98},
bibsource = {DBLP, http://dblp.uni-trier.de}
}


DBLP: Copyright ©1999 by Michael Ley (ley@uni-trier.de).