NiagaraCQ: A Scalable Continuous Query System for Internet Databases
Jianjun Chen (University of Wisconsin, now Microsoft), David J. DeWitt (University of Wisconsin, now Microsoft), Feng Tian (University of Wisconsin, now VMWare), Yuan Wang (University of Wisconsin, now Microsoft)
This paper from the SIGMOD 2000 Conference bridged from the world of continuous, or standing, queries against a changing stored database, to stream processing systems. NiagaraCQ was a pioneering system, the first to address the problem of the millions of overlapping queries that would need to be supported in a truly internet-scale system. It used relational-style operators to optimize a given set of continuous queries. Similar frameworks appeared in subsequent studies of stream databases, sensor databases, information delivery systems, and complex-event-processing (CEP) systems. The idea of dynamic optimization of continuous queries leveraging database operators (including dynamic query grouping and split) became a baseline for modern streaming data platforms. In summary, this paper helped open the new field of high-performance systems for continuous query processing, and was a strong force in shaping the following generations of stream processing systems.