SIGMOD 2023 Test-of-Time Award

Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management

Raul Castro Fernandez, Matteo Migliavacca, Evangelia Kalyvianaki, and Peter Pietzuch

This is the first paper that presents an abstraction for exposing internal operator state through a set of well-defined state management primitives, enabling an integrated and generic approach for dynamic scale out and recovery of stateful operators in stream processing systems. The work has led to a large body of follow-up work on scalable stream processing. Similar concepts and abstractions have been used by widely used stream processing platforms, making this paper a seminal paper in its field.