Guide to developing ToMCAT missions
ToMCAT aims to provide a flexible framework for studying human-machine teaming in the virtual world of Minecraft. It provides an API for constructing various scenarios in Minecraft that can act as focused tests of specific aspects of human-machine teaming as well as machine social intelligence. Following the parlance of Project Malmo, (the platform we build upon), we term these scenarios as ‘missions’.
In this section, we will go over how to design and implement a ToMCAT mission.
All file and directory paths are relative to the root of the
Building the environment
You have a couple of options when it comes to setting up the mission environment. - The first is to use a manually-designed and built (by yourself or someone
else) world. Then, specify the full path to the world folder and the starting position of the human player in the
Mission.ServerSection.ServerHandlers.FileWorldGeneratorsection of a Malmo-style XML specification. See
conf/xml_mission_specs/file_world.xmlfor an example of this approach.
The other option is to use ToMCAT’s
pro_genprocedural generation module (see
src/cpp/pro_genfor the source code and
tools/run_sessionfor example usage) to have more control over the generated mission environment. In an experiment context, it is usually desirable to have parametric control over the experimental conditions and stochasticity, both of which can be achieved with procedural generation.
Setting up the mission skeleton
After building the mission environment as described above, the next step is to define the mission behavior, - for example, the mission time limit, the behaviour of mobs, and handling in-game events of interest.
On the Java side, we will create some scaffolding code in the folder
To create a new mission called do the following steps outlined below. We’ll
call this mission
ExampleMission, and all the filepaths specified will be
folder. We will not go into all the details of the classes, but will point the
reader to the other missions in that directory (e.g.
ProceduralGenMission) as examples to follow.
Add a class called
ExampleMission.java. Specify the player’s initial position in the
Add a member to the
Mission.javafor the example mission.
Add a case in the switch statements for the
MissionFactory.javafor the example mission.
Create a class
If you want to be able to run the mission using the
in order to automatically set up A/V recording and the message bus, you’ll need
to add some code to the C++ side as well.
src/cpp/Mission.h: - Add a member to the MissionId enum. - Update
id_to_world_folder_mapwith the location of the world folder under
Update the description of the
src/cpp/runMission.cppwith a short description of
Add a line to the comments in
MAIN_MISSIONenvironment variable about
Defining event handlers
Given that you are implementing a custom mission, it’s likely that there are certain in-game events that you will want to (i) record for downstream analysis, (ii) trigger other in-game events, or both.
Events are recorded using JSON messages published to an MQTT message bus. In order to ensure valid JSON and reduce duplication of code, we define Java classes for the different types of events and use the Gson library to automatically serialize instances of those classes to JSON. The inheritance hierarchy of the classes helps us avoid code duplication for event types that are related by hypernymy/hyponymy, and thus share certain data fields.
The event classes reside in the
All event classes inherit from the
Event class, which has only two basic
attributes that are shared by all events:
In order to implement a new type of event, create a new Java class that
edu.arizona.tomcat.Event. The event should define private
data members that will be serialized by Gson. The constructor of the event may
or may not take an argument. Most of the implement event classes take one
argument, i.e. one of the event types that are published by Forge. The
Forge API documentation contains descriptions of all the possible events
that can be subscribed to, as well as the descriptions of the block and entity
types involved in the event.
Once you’ve created the event class, you can implement a handler for the event
in either your mission class (e.g. in
ExampleMission.java), or in the
ForgeEventHandler.java class. Most events that have been implemented so far
are handled in the
ForgeEventHandler class, which is then utilized in the
*Mission.java class files, since they are fairly general and not
specific to a particular mission.
For an example of handling a mission-specific event, see the
handler method in
Mission.java. For examples of non mission-specific event
handlers, see the
If you are implementing an event (see here for a list of events that are
potentially interesting for ToMCAT research) that needs to be published to the
message bus (this is most likely the case), then you’ll need to use the
MqttService class singleton class. The class has an overloaded method,
publish, which can publish either an object or a string - in the case of an
object, it is automatically serialized to JSON. The
publish method also
takes a second string-type argument, which is the topic that the message
corresponding to the event should be published to. You can see the currently
implemented topics and message data models here.
If you add an event that corresponds to a published message, you’ll also need
to add the message schema and topic for that event to
following the pattern of the other messages and topics that are already in