Welcome to ToMCAT
Current state-of-the art approaches to designing artificially intelligent (AI) teammates address some of the capabilities that they must have, including inferring the internal states of other agents solving problems collaboratively with them, and communicating with them in a socially-aware manner. To be a truly effective teammate, an AI agent must possess all of these capabilities, but no current approach combines them. To address this, we propose a Theory of Mind-based Cognitive Architecture for Teams (ToMCAT) with these capabilities.
ToMCAT will be tested in the virtual environment of Minecraft alongside human players. ToMCAT is comprised of a set of local agents (one for each human teammate) equipped with cameras and microphones to capture facial expressions and speech, as well as virtual sensors that record the local environment, the actions performed by human teammates, and chat exchanges between them. The local agents communicate with their respective humans, as well as with a global agent that performs coordination and global team optimization.
This site contains documentation and resources on the overall project, as well as the C++ API and links to the Java API and other ToMCAT components.
The ToMCAT project is funded by a 4.5 year, $7.5M DARPA grant as part of the Artificial Social Intelligence for Successful Teams (ASIST) program.
The images below show a few example screenshots from our inaugural ‘mission’, the Zombie Invasion.
|
|
| (i) Entrance to the besieged village | (ii) Fighting a zombie |
|
|
| (iii) Player's progress in the mission | (iv) Self-report: affect towards the mission. |
Contents:
The Github repository for ToMCAT can be found here.
Presentations