The ACM SIGMOD Comprehensive Reproducibility Award recognizes the best papers in terms of reproducibility. Up to three comprehensively reproduced papers are carefully selected and the awards are presented during the awards session of the SIGMOD conference (next year). Each award comes with a $750 honorarium. The criteria for the award broadly include: (i) coverage, (ii) ease of reproducibility, (iii) flexibility, and (iv) portability.
Specifically, the award recognizes reproducible papers that did the extra mile and provided artifacts that are fully and easily reproducible and offer increased flexibility (being able to add workloads, queries, data and exhibiting similar behavior with published results) and portability (ideally, portable to more than one operating systems and across versions)
Winners of 2022
Awarded to the most Comprehensively Reproduced Paper(s) of ACM SIGMOD 2021.
Conformance Constraint Discovery: Measuring Trust in Data-Driven Systems
by Anna Fariha (University of Massachusetts), Ashish Tiwari (Microsoft), Arjun Radhakrishna (Microsoft), Sumit Gulwani (Microsoft), Alexandra Meliou (University of Massachusetts)
Verified by: Maria Daltayanni, University of San Francisco
Winners of 2021
Awarded to Most Reproducible Papers of ACM SIGMOD 2020.
Functional-Style SQL UDFs With a Capital ‘F’
by Christian Duta, Torsten Grust
Verified by: Dmytro Bogatov, Boston University
Factorized Graph Representations for Semi-Supervised Learning from Sparse Data
by Krishna Kumar P., Paul Langton, Wolfgang Gatterbauer
Verified by: Asterios Katsifodimos, TU Delft
QueryVis: Logic-based Diagrams help Users Understand Complicated SQL Queries Faster
by Aristotelis Leventidis, Jiahui Zhang, Cody Dunne, Wolfgang Gatterbauer, H.V. Jagadish, Mirek Riedewald
Verified by: Raul Castro Fernandez, The University of Chicago
Winners of 2020
Awarded to Most Reproducible Papers of ACM SIGMOD 2019.
Raha: A Configuration-Free Error Detection System
by Mohammad Mahdavi, Ziawasch Abedjan, Raul Castro Fernandez, Samuel Madden, Mourad Ouzzani, Michael Stonebraker, Nan Tang
Verified by: Subarna Chatterjee, Harvard University
Uncertainty Annotated Databases – A Lightweight Approach for Approximating Certain Answers
by Su Feng, Aaron Huber, Boris Glavic, Oliver Kennedy
Verified by: Siqiang Luo, Harvard University
Winners of 2019
Awarded to Most Reproducible Papers of ACM SIGMOD 2018.
Randomized Algorithms Accelerated over CPU-GPU for Ultra-High Dimensional Similarity Search
by Yiqiu Wang, Anshumali Shrivastava, Jonathan Wang, Junghee Ryu
Verified by: Manos Athanassoulis and Subhadeep Sarkar, Boston University
Adaptive Optimization of Very Large Join Queries
by Thomas Neumann, Bernhard Radke
Verified by: Wilson Qin and Abdul Wasay, Harvard University
Robust Entity Resolution using Random Graphs
by Sainyam Galhotra, Donatella Firmani, Barna Saha, Divesh Srivastava
Verified by: Wilson Qin and Abdul Wasay, Harvard University
Winners of 2018
Awarded to Most Reproducible Papers of ACM SIGMOD 2017.
Transaction Repair for Multi-Version Concurrency Control
by Mohammad Dashti (EPFL), Sachin Basil John (EPFL), Amir Shaikhha (EPFL), Christoph Koch (EPFL)
Verified by: Stratos Idreos, Abdul Wasay and Wilson Qin, Harvard University
Data Canopy: Accelerating Exploratory Statistical Analysis
by Abdul Wasay (Harvard), Xinding Wei (Harvard), Niv Dayan (Harvard), Stratos Idreos (Harvard)
Verified by: Peter Triantafillou, University of Glasgow
Debunking the Myths of Influence Maximization: An In-Depth Benchmarking Study
by Akhil Arora (Xerox Research Centre India), Sainyam Galhotra (University of Massachusetts, Amherst), Sayan Ranu (Indian Institute of Technology, Delhi)
Verified by: Dan Olteanu, Oxford
Winners of 2017
Awarded to Most Reproducible Papers of ACM SIGMOD 2016.
Generating Preview Tables for Entity Graphs
by Ning Yan, Sona Hasani, Abolfazl Asudeh, Chengkai Li
Verified by: Hideaki Kimura
SQLShare: Results from a Multi-Year SQL-as-a-Service Experiment
by Shrainik Jain, Dominik Moritz, Daniel Halperin, Bill Howe, Ed Lazowska
Verified by: Juliana Freire, Fernando Seabra Chirigati, Tuan-Anh Hoang-Vu
Data Polygamy: The Many-Many Relationships among Urban Spatio-Temporal Data Sets
by Fernando Chirigati, Harish Doraiswamy, Theodoros Damoulas, Juliana Freire
Verified by: Azza Abouzied
UpBit: Scalable In-Memory Updatable Bitmap Indexing
by Manos Athanassoulis, Zheng Yan, Stratos Idreos
Verified by: Peter Triantafillou, George Sfakianakis