Mary Czerwinski, Eric Horvitz, and Ed Cutrell
Adaptive Systems & Interaction
Microsoft Research
Redmond, Washington 98052
Access pdf file.
Abstract
This paper explores a new approach to gauging users’ difficulties with tasks, interfaces, and situations we refer to as subjective duration assessment. The approach, adapted from results described in the interruption literature in psychology, centers on the use of time estimation to characterize performance. We introduce a metric, named relative subjective duration (RSD) that provides a means for probing the difficulty that users have with performing tasks—without directly asking users about the difficulty We do this in order to avoid the inherent bias toward the positive end of the scale typically seen in user satisfaction ratings after usability studies. It has been shown previously that when engaging tasks are interrupted, participants tend to overestimate how long those tasks take when compared to actual task times. Conversely, tasks that are completed tend to be underestimated in terms of the overall task times. We have explored the value of time estimation as a metric for evaluating task performance in HCI. Our hypothesis was that the duration of activity on tasks that are halted before completion would be overestimated, because participants were not able to complete them on their own, while the duration of activity on tasks completed successfully would be underestimated. A user study of a well-known Internet browser explored the efficacy of the metric. Our results show that within deployment constraints, RSD can be a valuable tool for HCI research.
Keywords: Usability, time perception, time estimation, metrics, empirical studies, relative subject duration (RSD).
In: Czerwinski, M., Horvitz, E. and Cutrell, E. (2001). Subjective Duration Assessment: An Implicit Probe for Software Usability. Proceedings of IHM-HCI 2001, Lille, France, September, 2001, pp. 167-170.