Experience Sampling for Building Predictive User Models: A Comparative Study

Ashish Kapoor and Eric Horvitz

Microsoft Research
Redmond, Washington 98052-6399

Access pdf.

Abstract:

Experience sampling has been employed for decades to collect assessments of subjects’ intentions, needs, and affective states. In recent years, investigators have employed automated experience sampling to collect data to build predictive user models. To date, most procedures have relied on random sampling or simple heuristics. We perform a comparative analysis of several automated strategies for guiding experience sampling, spanning a spectrum of sophistication, from a random sampling procedure to increasingly sophisticated active learning. The more sophisticated methods take a decision-theoretic approach, centering on the computation of the expected value of information of a probe, weighing the cost of the short-term disruptiveness of probes with their benefits in enhancing the long-term performance of predictive models. We test the different approaches in a field study, focused on the task of learning predictive models of the cost of interruption.

Keywords: Experience sampling, selective sampling, interruptions, active learning, user modeling

In: Proceedings of CHI 2008, Florence, Italy, April 2008.

Related Background

• A. Kapoor and E. Horvitz. On Discarding, Caching, and Recalling Samples in Active Learning, Proceedings of the Conference on Uncertainty and Artificial Intelligence 2007, AUAI Press, July 2007.
  
  
• A. Kapoor, E. Horvitz, and S. Basu. Selective Supervision: Guiding Supervised Learning with Decision-Theoretic Active Learning, Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, January 2007.

Back to Eric Horvitz's home page.