On evaluation of adaptive topic tracking systems Tamer Elsayed and Douglas W. Oard   View Paper (PDF)   Return to Posters Abstract Summative evaluation methods for supervised adaptive topic tracking systems convolve the effect of system decisions on present utility with the effect on future utility. This paper describes a new formative evaluation approach that focuses on future utility for use in the design stage of adaptive systems. Topic model quality is assessed at a predefined set of points using a fixed document set to enhance comparability. Experiments using a vector-space topic tracking system illustrate the utility of this approach to formative evaluation. ©2006 Association for Computing Machinery