AAAI 2022 Fall Symposium: Lessons Learned for Autonomous Assessment of Machine Abilities (LLAAMA)

Dates/Location: 17-19 November, Arlington, VA


Modern civilian and military systems have created a demand for sophisticated intelligent autonomous machines capable of operating in uncertain dynamic environments. Such systems are realizable thanks in large part to major advances in perception and decision-making techniques, which in turn have been propelled forward by modern machine learning tools. However, these newer forms of intelligent autonomy raise questions about when/how communication of the operational intent and assessments of actual vs. supposed capabilities of autonomous agents impact overall performance.  

This symposium session will examine possibilities for enabling intelligent autonomous systems to self-assess and communicate their ability to effectively execute assigned tasks, as well as reason about the overall limits of their competencies and maintain operability within those limits.  The symposium will bring together researchers working in this burgeoning area of research to share lessons learned, identify major theoretical and practical challenges encountered so far, and potential avenues for future research and real-world applications. 


We invite contributions from researchers in AI/expert systems, human factors, autonomous robotics and control/complex systems engineering, and other related disciplines that explore several key areas, including:

  • Applications and studies of competency self-assessments in field robotics and other real-world autonomous systems
  • Measures of competency for operational self-assessment by autonomous agents
  • AI/ML, uncertainty quantification, formal methods, and algorithmic meta-reasoning techniques to enable/support autonomous competency self-assessment     
  • Presentation and communication of machine generated competency self-assessments to human users/stakeholders
  • Techniques for evaluating the quality of machine generated competency self-assessments (e.g. correctness, completeness, fidelity, reliability).

This symposium will feature invited talks and contributed paper presentations by leading researchers and technical experts, as well as panel discussions and group breakout sessions focusing on the implementation of competency assessment in real autonomous systems.


Please submit one of the following types of submissions via the AAAI FSS-22 EasyChair site as a .pdf document (link to be posted here soon…). 

  • Regular papers (6-8 pages + references)
  • Position papers (2-4 pages + references)
  • Summary of previously published papers (1-2 pages)

Following review by and feedback from the symposium’s technical committee, authors of selected papers will be invited to give a short presentation for their contributions at the symposium.

Please use the following AAAI paper templates available here.

Invited Speakers and Program

To be posted…

Organizing Committee

Aastha Acharya, Nicholas Conlon, Nisar Ahmed (University of Colorado Boulder);

Rebecca Russell, Michael Crystal (Draper);

Brett Israelsen (Raytheon Technologies Research Center);

Ufuk Topcu (UT Austin);

Zhe Xu (Arizona State University);

Daniel Szafir (UNC)