Eric Horvitz: Selected publications

People, Machines, and Intelligence

Recent

• B. Wilder, E. Horvitz, E. Kamar. Learning to Complement Humans, IJCAI 2020.
  
  
• T. Hope, D. Downey, O. Etzioni, D.S. Weld, E. Horvitz, A Computational Inflection for Scientific Discovery, May 2022.
  
  
• R. Fogliato, S. Chappidi, M. Lungren, et al., Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging, ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT), June 2022.
  
  
• S. Singla, B. Nushi, S. Shah, E. Kamar, E. Horvitz. Understanding Failures of Deep Networks via Robust Feature Extraction, CVPR 2021.
  
  
• G. Bansal, B. Nushi, E. Kamar, et al. Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork, AAAI 2021.
  
  
• E. Kamar, S. Hacker, E. Horvitz. Combining Human and Machine Intelligence in Large-scale Crowdsourcing, AAMAS 2012.
  
  
• R. Ramakrishnan, E. Kamar, B. Nushi, D. Dey, J. Shah, E. Horvitz. Overcoming Blind Spots in the Real World: Leveraging Complementary Abilities for Joint Execution, AAAI 2019.
  
  
• G. Bansal, B. Nushi, E. Kamar, et al. Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff, AAAI 2019.
  
  
• S. Amershi, D. Weld, M. Vorvoreanu, A. Fourney, et al. Guidelines for Human-AI Interaction, CHI 2019.
  
  
• B. Nushi, E. Kamar, E. Horvitz. Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure, HCOMP 2018.
  
  
• D. Bohus, C.W. Saw, E. Horvitz. Directions Robot: In-the-Wild Experiences and Lessons Learned, AAMAS 2014.
  
  
• S. Jegelka, A. Kapoor, and E. Horvitz. An Interactive Approach to Solving Correspondence Problems, Intl Journal of Computer Vision, 2013.
  
  
• A. Kapoor, B. Lee, D. Tan, E. Horvitz. Performance and Preferences: Interactive Refinement of Machine Learning Procedures, AAAI 2012.
  
  
• D. Bohus and E. Horvitz. Decisions about Turns in Multiparty Conversation: From Perception to Action, ICMI 2011. (video)
  
  
• A. Kapoor, B. Lee, D. Tan, E. Horvitz. Interactive Optimization for Steering Classification, CHI 2010.
  
  
• D. Shahaf and E. Horvitz. Generalized Task Markets for Human and Machine Computation, AAAI 2010.
  
  
• D. Bohus and E. Horvitz. Models for Multiparty Engagement in Open-World Dialog, Sigdial 2009. video
  

Earlier

• E. Horvitz. Principles of Mixed-Initiative User Interfaces. CHI 1999 video, 1999 tv debut
  
  
• E. Horvitz and T. Paek. Complementary Computing: Policies for Transferring Callers from Dialog Systems to Human Receptionists. User Modeling and User Adapted Interaction 17 (2007).
  
  
• E. Horvitz and M Barry. Display of Information for Time-Critical Decision Making. UAI 1995.
  
  
• E. Horvitz, J. Breese, D. Heckerman, D. Hovel, and K. Rommelse. The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. UAI 1998, pp. 256-265.   video   more info.
  
  
• E. Horvitz, A. Jacobs, D. Hovel. Attention-Sensitive Alerting, UAI 1999, pp. 305-313. pdf, video
  
  
• E. Horvitz, C. M. Kadie, T. Paek, D. Hovel. Models of Attention in Computing and Communications: From Principles to Applications, CACM 46(3):52-59, March 2003.   NYTimes article   Video: Directions & futures   Video: Priorities & Notification Platform   photo.
  
  
• E. Horvitz, S. Dumais, P. Koch. Learning Predictive Models of Memory Landmarks, Cognitive Science 2004 (CogSci). video
  
  
• E. Horvitz and J. Apacible. Learning and Reasoning about Interruption. ICMI 2003.
  
  
• E. Horvitz, D. Heckerman, B. Nathwani, L.M. Fagan, The Use of a Heuristic Problem-Solving Hierarchy to Facilitate the Explanation of Hypothesis-Directed Reasoning, Medinfo 1986, pp. 27-31.
  
  
• E. Horvitz. Reflections on Challenges and Promises of Mixed-Initiative Interaction, AI Magazine 28 (2007).
  
  
• N. Oliver and E. Horvitz. Selective Perception Policies for Limiting Computation in Multimodal Systems: A Comparative Analysis, ICMI 2003 (also appears: Lectures Notes in Computer Science. Springer Verlag, Jan. 2005).
  
  
• N. Oliver, A. Garg, and Eric Horvitz. Layered Representations for Learning and Inferring Office Activity from Multiple Sensory Channels, Computer Vision and Image Understanding (CVIU), 96(2004), pp. 163-180. Presentation at IJCAI 2001.

AI Principles, Methods, and Systems

Recent

• S.J. Gershman, E.J. Horvitz, J.B. Tenenbaum. Computational Rationality: A Converging Paradigm for Intelligence in Brains, Minds, and Machines, 16 July 2015, Science 349. 273-278 (2015).
  
  
• A. Kolobov, Y. Peres, C. Lu, E. Horvitz. Staying up to Date with Online Content Changes Using Reinforcement Learning for Scheduling, NeurIPS 2019.
  
  
• E. Kamar and E. Horvitz. Light at the End of the Tunnel: A Monte Carlo Approach to Computing Value of Information, AAMAS 2013.
  
  
• H. Aly, J. Krumm, G. Ranade, E. Horvitz. Computing Value of Spatiotemporal Information, CACM 2020.
  
  
• M. Srivastava, B. Nushi, E. Kamar, S. Shah, E. Horvitz. Backward Compatibility in Machine Learning Systems, KDD 2020.
  
  
• A. Modi, D. Dey, A. Agarwal, A. Swaminathan, B. Nushi, S. Andrist, E. Horvitz. Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations, AAAI 2020.
  
  
• A. Grover, A. Kapoor, E. Horvitz. A Deep Hybrid Model for Weather Forecasting. KDD 2015.
  
  
• R. Ramakrishnan, E. Kamar, D. Dey, J. Shah, E. Horvitz. Discovering Blind Spots in Reinforcement Learning, AAMAS 2018.
  
  
• J. Krumm and E. Horvitz. Traffic Updates: Saying a Lot While Revealing a Little, AAAI 2019.
  
  
• Y. Azar, E. Horvitz, E. Lubetzky, Y. Peres, D. Shahaf. Tractable near-optimal policies for crawling, PNAS 2018. (pdf)
  
  
• H. Lakkaraju, E. Kamar, R. Caruana, E. Horvitz. Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration, AAAI 2017.
  
  
• J.C. Krumm and E. Horvitz. Risk-Aware Planning: Methods and Case Study for Safe Driving Routes, IAAI 2017.
  
  
• C.H. Lin, A. Kolobov, E. Kamar, E. Horvitz. Metareasoning for Planning Under Uncertainty, IJCAI 2015.
  
  
• A. Kapoor and E. Horvitz. Breaking Boundaries: Active Information Acquisition Across Learning and Diagnosis, NIPS 2009.
  

Earlier

• E. Horvitz. Principles and Applications of Continual Computation, AI Journal, 126:159-196 (2001).
  
  
• E. Horvitz, Y. Ruan, C. Gomes, H. Kautz, B. Selman, D. M. Chickering. A Bayesian Approach to Tackling Hard Computational Problems. UAI 2001, pp. 235-244. Slides.
  
  
• P. Dagum and E. Horvitz. A Bayesian Analysis of Simulation Algorithms for Inference in Belief Networks. Networks, 23:499-516, 1993.
  
  
• D. Shahaf and E. Horvitz. Investigations of Continual Computation, IJCAI 2009.
  
  
• F. Bach, D. Heckerman, E. Horvitz. Considering Cost Asymmetry in Learning Classifiers. Journal of Machine Learning Research, 7 (2006) 1713–1741.
  
  
• E. Horvitz and A. Klein, Utility-Based Abstraction and Categorization. UAI 1993, pp. 128-135.
  
  
• P. Dagum, A. Galper, and E. Horvitz. Dynamic Network Models for Forecasting, UAI 1992.
  
  
• E.J. Horvitz, J.S. Breese, M. Henrion, Decision Theory in Expert Systems and Artificial Intelligence, Journal of Approximate Reasoning, 2:247-302, 1988.
  
  
• E.J. Horvitz, G.F. Cooper, D.E. Heckerman, Reflection and action under scarce resources: Theoretical principles and empirical study. IJCAI 1989, pp. 1121-1127.
  
  
• D. Heckerman, J.S. Breese, E. Horvitz, The Compilation of Decision Models, UAI 1989, pp. 162-173.
  
  
• E.J. Horvitz, Reasoning under varying and uncertain resource constraints. AAAI 1988, pp. 111-116.
  
  
• E. Horvitz, Reasoning about Beliefs and Actions under Computational Resource Constraints, In: L. Kanal, et al. ed., Uncertainty in Artificial Intelligence 3, Elsevier, 1989, pp. 301-324. (Conference version: E. Horvitz, Reasoning about Beliefs and Actions under Computational Resource Constraints, UAI 1987, pp. 429-444.
  
  
• E. Horvitz, J. Apacible, R. Sarin, and L. Liao. Prediction, Expectation, and Surprise: Methods, Designs, and Study of a Deployed Traffic Forecasting Service, UAI 2005.
  
  
• D. Azari, E. Horvitz, S. Dumais, E. Brill. Actions, Answers, and Uncertainty: A Decision Making Perspective on Web-Based Question Asking, Information Processing and Management, 40(5), 2004, pp. 849-868.
  
  
• P. N. Bennett, S. T. Dumais, and E. Horvitz. The Combination of Text Classifiers using Reliability Indicators. Information Retrieval.
  
  
• B. Selman, R. Brooks, T. Dean, E. Horvitz, T. Mitchell, N. Nilsson. Challenge Problems for Artificial Intelligence. AAAI, August 1996, pp. 1340-1345.
  

Biomedical Informatics

Recent

• U. Singer, K. Radinsky, E. Horvitz. On Biases of Attention in Scientific Discovery, Journal of Bioinformatics, December 2020. At J. Bioinformatics.
  
  
• J. Wiens, J. Guttag, and E. Horvitz. Patient Risk Stratification with Time-Varying Parameters: A Multitask Learning Approach. JMLR, April 2016.
  
  
• D.H. Lee, M. Yetisgen, L. Vanderwende, E. Horvitz. Predicting Severe Clinical Events by Learning about Life-Saving Actions and Outcomes using Distant Supervision, J. Biomedical Informatics 2020.
  
  
• M. Bayati, M. Braverman, M. Gillam, K.M. Mack, G. Ruiz, M.S. Smith, E. Horvitz. Data-Driven Decisions for Reducing Readmissions for Heart Failure: General Methodology and Case Study. PLOS One Medicine. October 2014.
  
  
• J. Wiens, Wayne N. Campbell, Ella S. Franklin, J. Guttag, E. Horvitz. Learning Data-Driven Patient Risk Stratification Models for Clostridium difficile. Open Forum Infectious Diseases Advance Access, June 2014.
  
  
• J. Wiens, J. Guttag, and E. Horvitz. A Study in Transfer Learning: Leveraging Data from Multiple Hospitals to Enhance Hospital-Specific Predictions, JAMIA: 0:1–8 (2014).
  
  
• R.W. White, N.P. Tatonetti, N. H. Shah, R.B. Altman, E. Horvitz. Web-Scale Pharmacovigilance: Listening to Signals from the Crowd. JAMIA, March 2013.
  
  
• J. Wiens, J. Guttag, E. Horvitz. Patient Risk Stratification for Hospital-Associated C. Diff as a Time-Series Classification Task, NIPS 2012.
  
  
• E. Horvitz. From Data to Predictions and Decisions: Enabling Evidence-Based Healthcare, Computing Community Consortium, September 2010.
  
  
• R.W. White and E. Horvitz. Cyberchondria: Studies of the Escalation of Medical Concerns in Web Search. ACM Transactions on Information Systems, 27(4), Article 23, November 2009, DOI 101145/1629096.1629101.
  
  
• R.W. White, P. Murali Doraiswamy, and E. Horvitz. Detecting Neurodegenerative Disorders from Web Search Signals. Nature Digital Medicine, April 2018.
  
  
• E. Horvitz, Perspective: The Future of Biomedical Informatics: Bottlenecks and Opportunities, In: E.H. Shortliffe, J.J. Cimino, et. al, Biomedical Informatics: Computer Applications in Health Care and Biomedicine, Springer, 2021.
  
  
• R. W. White and E. Horvitz. Early Signs of Lung Carcinoma in Web Search Logs: Findings and Implications, JAMA Oncology, November 2016.
  
  
• T. Althoff, E. Horvitz, R.W. White, J. Zeitzer, Population-Scale Study of Sleep and Performance, WWW 2017.
  
  
• J. Paparrizos, R.W. White, E. Horvitz. Screening for Pancreatic Adenocarcinoma using Signals from Web Search Logs: Feasibility Study and Results, J. Oncology Practice, June 7, 2016. doi: 10.1200/JOP.2015.010504.
  
  
• M.J. Paul, R.W. White, E. Horvitz. Search and Breast Cancer: On Episodic Shifts of Attention over Life Histories of an Illness, ACM Transactions on the Web, 2016.
  
  
• R.W. White, S. Wang, A. Pant, R. Harpaz, P. Shukla, W. Sun, W. DuMouchel, E. Horvitz. Early Identification of Adverse Drug Reactions from Search Log Data, J. Biomedical Informatics. 59:42–48, 2015.
  
  

Earlier

• E. Horvitz and A. Seiver. Time-Critical Action: Representations and Application. UAI 1997.
  
  
• E. Horvitz and G. Rutledge. Time-Dependent Utility and Action Under Uncertainty. UAI 1991, pp. 151-158.
  
  
• D. E. Heckerman, E. J. Horvitz, and B. N. Nathwani. Toward Normative Expert Systems: Part I The Pathfinder Project. Methods of Information in Medicine, 31:90-105 (1992).
  
  
• E. Horvitz, D. Heckerman, B. Nathwani, L.M. Fagan, The use of a heuristic problem-solving hierarchy to facilitate the explanation of hypothesis-directed reasoning, Medinfo 1986, pp. 27-31.
  
  
• E.J. Horvitz, D.E. Heckerman, B.N. Nathwani, L.M. Fagan, Diagnostic Strategies in the Hypothesis-Directed Pathfinder System, In: First IEEE Conference on Artificial Intelligence Applications, December 1984, pp. 630-636.
  

Computing, People, and Society

• E. Horvitz. AI, People, and Society, Science 357:6346, pp. 7. (2017) DOI: 10.1126/science.aao2466
  
  
• E. Horvitz and D. Mulligan. Data, privacy, and the greater good, 16 July 2015, Science 349. pp. 253-255 (2015).
  
  
• E. Horvitz, Reflections on the Status and Future of Artificial Intelligence, Testimony Before the United States Senate, Hearing on the Dawn of Artificial Intelligence, Committee on Commerce Subcommitte on Space, Science, and Competitiveness, November 30, 2016.
  
  
• T.G. Dietterich and E.J. Horvitz, Rise of Concerns about AI: Reflections and Directions. CACM 58:10, pp. 38-40 10.1145/2770869
  
  
• E. Horvitz, M. Clyburn, E. Felten, T. LeBlanc, Caution Ahead: Navigating Risks to Freedoms Posed by AI. The Hill, May 17, 2021.
  
  
• E. Fast and E. Horvitz. Long-Term Trends in the Public Perception of Artificial Intelligence, AAAI 2017.
  
  
• A. Howard, C. Zhang, E. Horvitz. Addressing Bias in Machine Learning Algorithms: A Pilot Study on Emotion Recognition for Intelligent Systems. IEEE Workshop on Advanced Robotics and its Social Impacts, 2017 DOI: 10.1109/ARSO.2017.8025197
  
  
• J. Suh, E. Horvitz, R.W. White, T. Althoff, Population-Scale Study of Human Needs During the COVID-19 Pandemic: Analysis and Implications, WSDM 2021.
  
  
• A. Krause and E. Horvitz. A Utility-theoretic Approach to Privacy in Online Services, JAIR, 39 (2010) 633-662.
  
  
• A. Singla, E. Horvitz, E. Kamar, R.W. White. Stochastic Privacy, AAAI 2014.
  
  
• J. Benaloh, M. Chase, E. Horvitz, K. Lauter. Patient Controlled Encryption: Ensuring Privacy in Electronic Medical Records, ACM CCSW 2009, Chicago, IL, November, 2009.
  
  
• A. Krause, E. Horvitz, A. Kansal, F. Zhao. Toward Community Sensing, IPSN 2008.
  
  
• J. Aythora, et al., Multi-stakeholder Media Provenance Management to Counter Synthetic Media Risks in News Publishing, IBC 2020 (video.)
  
  
• P. England, H.S. Malvar, E. Horvitz, J.W. Stokes, C. Fournet, et al., AMP: Authentication of Media via Provenance, ACM Multimedia Systems 2021.
  
  
• A. Spangher, G. Ranade, B. Nushi, A. Fourney, E. Horvitz. Characterizing Search-Engine Traffic to Internet Research Agency Web Properties, Web Conf. 2020.
  
  
• R. Boyd, A. Spangher, A. Fourney, B. Nushi, G. Ranade, J. Pennebaker, E. Horvitz. Characterizing the Internet Research Agency's Social Media Operations During the 2016 U.S. Presidential Election using Linguistic Analyses, PsyArXiv Preprints 2018. doi: 10.31234/osf.io/ajh2q
  
  
• A. Fourney, M.Z. Racz, G. Ranade, M. Mobius, E. Horvitz. Geographic and Temporal Trends in Fake News Consumption During the 2016 US Presidential Election, CIKM 2017.
  
  
• Eric Horvitz, One-Hundred Year Study on Artificial Intelligence: Reflections and Framing, One Hundred Year Study on Artifical Intelligence, Stanford University 2014.
  
  
• E. Horvitz, J. Young, R.G. Elluru, C. Howell. Key Considerations for the Responsible Development and Fielding of Artificial Intelligence, National Security Commission on Artificial Intelligence (NSCAI), April 2021. Abridged version, podcast highlight
  
  
• Safra Catz, Steve Chien, Mignon Clyburn, Chris Darby, Kenneth Ford, José-Marie Griffiths Eric Horvitz Andrew Jassy, Gilman Louie, William Mark, Jason Matheny, Katharina McFarland, Andrew Moore, Eric Schmidt, Robert Work. Report of the National Security Commission on Artificial Intelligence, National Security Commission on Artificial Intelligence (NSCAI), March 2021.
  
  
• AAAI Presidential Panel on Long-Term AI Futures, Association for the Advancement of AI (AAAI), 2008-2009.

Foundations

Studies in Bounded Rationality

• Introducing bounded optimality and anytime algorithm paradigm:
     E. Horvitz, Reasoning about Beliefs and Actions under Computational Resource Constraints, UAI 1987, pp. 429-444. (Extended book version.)
  
  
• On principles of metareasoning:
     E.J. Horvitz, G.F. Cooper, D.E. Heckerman, Reflection and action under scarce resources: Theoretical principles and empirical study. IJCAI 1989, pp. 1121-1127.
  
  
• On the cost of thinking:
     E.J. Horvitz, Reasoning under varying and uncertain resource constraints. AAAI 1988, pp. 111-116.
  
  
• Streaming intelligence. Principles of prescient computing to handle streams of problems:
     E. Horvitz. Principles and Applications of Continual Computation, AI Journal, 126:159-196 (2001).
  
  
• Thinking about thinking: Models and optimization
     E. Horvitz and J.S. Breese, Ideal Partition of Resources for Metareasoning. arXiv:2110.09624, February 1990.
  
  
• From reasoning to reflex:
     D. Heckerman, J.S. Breese, E. Horvitz, The Compilation of Decision Models, UAI 1989, pp. 162-173.
  
  
• Computational rationality as convergent paradigm:
     S.J. Gershman, E.J. Horvitz, J.B. Tenenbaum. Computational Rationality: A Converging Paradigm for Intelligence in Brains, Minds, and Machines, 16 July 2015, Science 349. 273-278 (2015).

Human-AI Complementarity and Collaboration

• On mixed-initiative interaction:
     E. Horvitz. Principles of Mixed-Initiative User Interfaces. CHI 1999.
  
  
• On guiding the display of information:
     E. Horvitz and M Barry. Display of Information for Time-Critical Decision Making. UAI 1995.
  
  
• On inferring the goals and needs of software users:
     E. Horvitz, J. Breese, D. Heckerman, D. Hovel, and K. Rommelse. The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. UAI 1998, pp. 256-265.
  
  
• Models of information value and alerting:
     E. Horvitz, A. Jacobs, D. Hovel. Attention-Sensitive Alerting, UAI 1999, pp. 305-313.
  
  
• Machine learning and models of memory:
     E. Horvitz, S. Dumais, P. Koch. Learning Predictive Models of Memory Landmarks, Cognitive Science 2004 (CogSci). video
  
  
• Predicting the cost of interruption:
     E. Horvitz and J. Apacible. Learning and Reasoning about Interruption. ICMI 2003.
  
  
• Models of attention in computing:
     E. Horvitz, C. M. Kadie, T. Paek, D. Hovel. Models of Attention in Computing and Communications: From Principles to Applications, Communications of the ACM 46(3):52-59, March 2003.
  
  
• General task markets:
     D. Shahaf and E. Horvitz. Generalized Task Markets for Human and Machine Computation, AAAI 2010.
  
  
• Ideal transfer from AI to humans:
     E. Horvitz and T. Paek. Complementary Computing: Policies for Transferring Callers from Dialog Systems to Human Receptionists. User Modeling and User Adapted Interaction 17 (2007).
  
  
• Combining human and machine capabilities:
     E. Kamar, S. Hacker, E. Horvitz. Combining Human and Machine Intelligence in Large-scale Crowdsourcing, AAMAS 2012.
  
  
• On learning to complement humans:
     B. Wilder, E. Horvitz, E. Kamar. Learning to Complement Humans, IJCAI 2020.

Machine learning and combinatorial problems

• On harnessing machine learning to guide solution of NP-Hard problems:
     E. Horvitz, Y. Ruan, C. Gomes, H. Kautz, B. Selman, D. M. Chickering. A Bayesian Approach to Tackling Hard Computational Problems. UAI 2001, pp. 235-244. Slides.
  
  
• Optimal restart policies for theorem proving
     H. Kautz, E. Horvitz, Y. Ruan, C. Gomes, B. Selman. Dynamic Restart Policies. Proceedings of the Eighteenth National Conference on Artificial Intelligence, AAAI 2002.
  
  
• Computing value of information in planning
     E. Kamar and E. Horvitz. Light at the End of the Tunnel: A Monte Carlo Approach to Computing Value of Information, AAMAS 2013.