Data Cleaning in Microsoft SQL Server 2005 Surajit Chaudhuri, Kris Ganjam, Venky Ganti, Rahul Kapoor, Vivek R. Narasayya, and Theo Vassilakis   View Paper (PDF)   Return to Group 2 Demonstrations Abstract When collecting and combining data from various sources into a data warehouse, ensuring high data quality and consistency becomes a significant, often expensive, challenge. Common data quality problems include inconsistent data conventions amongst sources such as different abbreviations or synonyms; data entry errors such as spelling mistakes; missing, incomplete, outdated or otherwise incorrect attribute values. These data defects generally manifest themselves as foreign-key mismatches and approximately duplicate records, both of which make further data mining and decision support analyses either impossible or suspect. We demonstrate two new data cleansing operators, Fuzzy Lookup and Fuzzy Grouping, which address these problems in a scalable and domain-independent manner. These operators are implemented within Microsoft SQL Server 2005 Integration Services. Our demo will explain their functionality and highlight multiple realworld scenarios in which they can be used to achieve high data quality. ©2006 Association for Computing Machinery