Meeting on

Foundations of Augmented Cognition


August 15-16, 2001

National Academy of Sciences
2101 Constitution Avenue, NW
Washington, DC



  Eric Horvitz and Misha Pavel


Sponsoring organization:

Information Science and Technology Study Group (ISAT)

Defense Advanced Research Projects Agency



The ISAT group is charged with providing the Information Technology Office (ITO) of DARPA with continuing, independent assessments of information sciences and technologies.  The group identifies opportunities for innovations in computation, communications, and related applications, and performs strategic appraisal of technologies and overall new research directions for DARPA.  


At the ISAT Woods Hole meeting in the summer of 2000, the phrase augmented cognition was first used during deliberations of a DARPA working group exploring new directions for DARPA. Eric Horvitz used the term to define a potentially fruitful endeaver of research that would explore opportunities for developing principles and computational systems that support and extend human cognition by taking into explicit consideration well-characterized limitations in human cognition, spanning attention, memory, problem solving, and decision making. The proposed research included opportunities for developing fast-paced and fluid computational methods that could react to cognitive state sensed in a variety of ways, as well as behavioral and functional designs developed to complement limitations in human cognition. Research efforts on using Bayesian attentional models in prototypes at Microsoft Research and two earlier ISAT studies in the area of memory had highlighted the broad opportunities in this area. Low-hanging fruit included methods for managing interruption and recovery, assisting with multitasking, enhancing bandwidth via exploiting multiple channels, providing reminders or assistance at the right time, and addressing biases in judgment. He presented Augmented Cognition as a key direction to DARPA Director Frank Fernandez and program managers at a meeting in Woods Hole, MA in July 2000.


In distinction to related efforts in HCI that have drawn broadly on principles of human cognition and models of cognition as information processing to support design, augmented cognition focuses on the precise targeting of one or more specific cognitive bottlenecks within larger applications—and on opportunities to build out carefully from these specific components, potentially bringing together multiple approaches to cognitive support within single systems.


We can draw a variety of examples of opportunities for augmented cognition applications from different subspecialties in psychology.  For example, there is opportunity to leverage models and experimental results from cognitive psychology on divided attention and disruption in peripheral awareness and monitoring applications. Such results can be used to control if, how, and when information about a monitored system is presented to users focusing on another, more central task.  As another example, results from visual search and attention might be leveraged in designs for information display that have the ability to relay with increased efficiency the most critical information to users in time-critical settings. 


Beyond building and testing real-world applications, augmented cognition includes explicit efforts to enrich knowledge about cognitive mechanisms by taking advantage of new kinds of user studies during attempts to build, deploy, and test systems designed to address cognitive limitations.  For example, new psychological studies and results might be developed by studying how users react to different alerting systems or policies in specific contexts. As another example, it may be feasible to investigate how different visual representations of actions alternatives and uncertain outcomes might help in debiasing well-characterized errors of judgment in fusing multiple findings and decision making under uncertainty.


The goal of the ISAT study on Foundations of Augmented Cognition has been to take a pass at mapping out the key challenges and opportunities, as well as initial directions that promise to be successful or to provide valuable learning experiences.  The effort is aimed at further clarifying challenges and opportunities, and to support DARPA program management with an assessment of directions and thoughts on developing programs related to this topic. 


Additional Background


As follow up on the work on DARPA directions last summer, we held an workshop on directions for augmented cognition in Seattle in November, 2000, organized by Eric Horvitz, Lee Kollmorgen, and Dylan Schmorrow.  This meeting was followed up by an ISAT breakout group on augmented cognition during the adjacent Fall 2000 ISAT meeting.  Slides from the briefing created by this group are available via a link below.  The ISAT breakout group led to two ISAT efforts.  The first, is a formal 2001 study on “Mixed-Initiative Control and Monitoring within Human-'Bot Systems.” The second exploration is an incubation effort centering on an exploration of the terrain, set of opportunities with augmented cognition, referred to as the “Foundations of Augmented Cognition Technologies” incubation effort.  We are holding the conference at the National Academy of Science as part of this effort. We view the output of the ISAT study as providing support for the bidding and definition of one or more future DARPA programs in Augmented Cognition.We are coordinating with Dylan Schmorrow, a program manager at DARPA. Dylan has been pursuing possibilities for putting together a formal DARPA program focusing on some aspect of augmented cognition. During the coarse of the ISAT study, Dylan has pursued the funding of a set of seedling projects in this area. 


Several early presentations


Meeting Events

Meeting program


Breakout groups









Travel and lodging arrangements

Participants will receive a message from Rich Entlich on details about travel and lodging.


ISAT contact for arrangements:

Rich Entlich (

Institute for Defense Analyses

1801 North Beauregard Street

Alexandria, VA 22311-1772

703-845-6648 (tel.), 703-845-6848 (fax)


Briefing to ISAT, Woods Hole, August 2001