Welcome to BW4T-Matrx

This is our repo for agents for BW4T running on matrx

You can copy the code using git clone https://github.com/rsverhagen94/TU-Delft-Collaborative-AI-Trust.git

More info on matrx core ​http://docs.matrx-software.com/en/master/autoapi/matrx/world_builder/index.html#matrx.world_builder.WorldBuilder.add_agent

Usage

Installation

• git clone the project as above
• (optional) set up your virtual environment
• pip install -r requirements.txt

Setup

You can change the main.py script that contains the setup settings for the run.

main.py settings

• Edit main.py program with your favourite text editor
• If you added a new TeamXXAgent, import it
• locate the section containing the agents list, like this

  agents = [
      {'name':'agent1', 'botclass':BaseLineAgent, 'settings':{'slowdown':10}},
      {'name':'agent2', 'botclass':BaseLineAgent, 'settings':{}},
      {'name':'human', 'botclass':Human, 'settings':{}}
      ]
  
  

• Edit this section as needed. Each line is creating one bot. Names must be unique. Botclass must match the class name of the agent you need. Slowdown X means that this agent slows by a factor X, the minimum number is 1.
• save the file. Make sure it remains plain ASCI text and don't change the end-of-line markers.

Run

While running, do not open the csv log file that the session/tournament is writing to: doing this may block the file for writing and cause the session to halt with a write error.

Run Session

• run python main.py
• In a web browser (eg safari or firefox) go to ​http://localhost:3000
• Go to god mode and start the simulation by pressing the white triangle at the top
• The log file is written to world1 directory.

Human agent

A human bot is controlled through the web browser. Go to ​http://localhost:3000 and select the humanbot you want to control. Now you can navigate the humanbot using the keyboard:

keyboard letter	effect
WASD	move agent
Q	grab an object
E	to drop object
R	to open a door
F	to close it

Humans can also send and receive messages. Click on the balloons-icon at the top right. An additional area will appear at the right or at the bottom of the screen (notice, in many browsers this remains invisible because they hide the scroll bar). You can type your own messages and select agent(s) to address to.

Implementing your agent

To implement an agent for BW4T, all you need is to create a class that extends the bw4t/BW4TBrain. To do this, you must extend filter_bw4t_observations and decide_on_bw4t_action of bw4t/BW4TBaseLineAgent. So to implement your agent, you only need to extend the bw4t/BW4TBrain and bw4t/BW4TBaseLineAgent.

If you implement init, you must call super().init(settings) with the original settings as first call in your init function.

If you implement initialize, you must call super().initialize() with the original settings as first call in your initialize function.

You cannot override get_log_data because we use it already for logging the outcomes of the game.

With that done, you can import and use your agent in main.py, check #Setup.

The following sections give some details about the two functions to implement.

filter_bw4t_observations

This class is called every tick, with the new State object. You should process here all observations, because observations may be visible only for one tick.

Normally this function returns the state. If you return a modified state here, then decide_on_bw4t_actions will receive this modified state.

decide_on_bw4t_action

This function is called every time your agent can take an action. This may not be every tick. The decide_on_bw4t_action receives the state as parameter. The agent code decides on this and returns what the next action is, in the form of a tuple (action, params).

The following actions and params are possible in BW4T:

Action	required params
'OpenDoorAction'	'door_range':1, 'object_id':doorId
'MoveNorth' 'MoveEast' 'MoveSouth' 'MoveWest'
GrabObject	'object_id':blockId
DropObject	'object_id':blockId
None (do nothing)

BW4TBrain

All agents for BW4T are required to extend our BW4TBrain. Implementing a BW4TBrain is slightly different from implementing a normal AgentBrain (the default superclass for agents):

• decide_on_bw4t_action replaces decide_on_action. BW4T agents must not override decide_on_action
• filter_bw4t_observations replaces filter_observations. Agents must not override filter_observations
• agent action parameters are restricted, they can not set 'remove_range', 'grab_range', 'door_range' and 'action_duration'.

The following values are enforced for the parameters for all actions

action parameter	value
grab_range	1
action_duration	set by the agent slowdown setting

State info

When implementing your agent, you receive a State object. It contains a dictionary representing a gridworld state. The gridworld is a 2D grid with a set of objects/tiles, each at a given 'location' on the grid. Some tiles like walls are fixed, some tiles like bots and blocks can be moved The keys are unique IDs from the Matrx system, and the values again are dictionaries. The values dictionary contents depend on the type of tile. Typically agents will receive something like this:

dict: {
'agent_0_in_team_0_344': 
  {
    'isAgent': True, 
    'team': '', 
    'name': 'Agent 0 in Team 0', 
    'obj_id': 'agent_0_in_team_0_344', 
    'location': (1, 1), 
    'is_movable': True, 
    'action_set': ['MoveSouthWest', 'OpenDoorAction', 'MoveWest', 'MoveSouth', 'GrabObject', 'MoveEast', 'MoveNorthEast', 'DropObject', 'MoveNorth', 'RemoveObject', 'MoveNorthWest', 'Move', 'MoveSouthEast', 'CloseDoorAction'], 
    'carried_by': [], 
    'is_human_agent': False, 
    'is_traversable': True, 
    'class_inheritance': ['BlockWorldAgent', 'AgentBrain', 'object'],
    'is_blocked_by_action': False, 
    'is_carrying': [], 
    'sense_capability': {<class 'matrx.objects.agent_body.AgentBody'>: 2, <class 'builder.CollectableBlock'>: 2, '*': inf}, 
    'visualization': {'size': 1.0, 'shape': 1, 'colour': '#92f441', 'depth': 100, 'opacity': 1.0, 'show_busy': False}, 
    'current_action': None, 
    'current_action_args': {}, 
    'current_action_duration': -1000000, 
    'current_action_started_at_tick': -1000000
  }, 
  'human_0_in_team_0_345':{
    'isAgent': True,
    'key_action_map': {'w': 'MoveNorth', 'd': 'MoveEast', 's': 'MoveSouth', 'a': 'MoveWest', 'q': 'GrabObject', 'e': 'DropObject', 'r': 'OpenDoorAction', 'f': 'CloseDoorAction'},
    'team': '',
    'name': 'Human 0 in Team 0',
    'obj_id': 'human_0_in_team_0_346',
    'location': (3, 1),
    'is_movable': True,
    'action_set': ['Move', 'MoveWest', 'GrabObject', 'MoveSouthEast', 'MoveNorthWest', 'MoveNorthEast', 'MoveEast', 'MoveSouth', 'OpenDoorAction', 'RemoveObject', 'MoveSouthWest', 'DropObject', 'MoveNorth', 'CloseDoorAction'],
    'carried_by': [],
    'is_human_agent': True,
    'is_traversable': False,
    'class_inheritance': ['HumanAgentBrain', 'AgentBrain', 'object'],
    'is_blocked_by_action': False,
    'is_carrying': [],
    'sense_capability': {<class 'matrx.objects.agent_body.AgentBody'>: 2, <class 'builder.CollectableBlock'>: 2, '*': inf},
    'visualization': {'size': 1.0, 'shape': 1, 'colour': '#92f441', 'depth': 100, 'opacity': 1.0, 'show_busy': False},
    'current_action': None,
    'current_action_args': {},
    'current_action_duration': -1000000,
    'current_action_started_at_tick': -1000000
  },
  'world_bounds_-_wall@(23,_4)_0': {
    'room_name': 'world_bounds', 
    'name': 'world_bounds - wall@(23, 4)', 
    'obj_id': 'world_bounds_-_wall@(23,_4)_0', 
    'location': (23, 4), 
    'is_movable': False,
    'carried_by': [],
    'is_traversable': False,
    'class_inheritance': ['Wall', 'EnvObject', 'object'],
    'visualization': {'size': 1.0, 'shape': 0, 'colour': '#000000', 'depth': 80, 'opacity': 1.0}
  }

  'world_bounds_-_wall@(0,_5)_0': {'room_name': 'world_bounds', ...}
  'room_0_-_wall@(4,_4)_94': {'room_name': 'room_0', name:...}
  'room_0_-_wall@(5,_7)_102': {
    'room_name': 'room_0', 
    'name': 'room_0 - wall@(5, 7)', 
    'obj_id': 'room_0_-_wall@(5,_7)_102', 
    'location': (5, 7), 
    'is_movable': False, 
    'carried_by': [], 
    'is_traversable': False, 
    'class_inheritance': ['Wall', 'EnvObject', 'object'],
    'visualization': {'size': 1.0, 'shape': 0, 'colour': '#8a8a8a', 'depth': 80, 'opacity': 1.0}
  }
  'room_0_-_wall@(6,_4)_101'
  ....
  'room_0_-_door@(7,_7)_109': {
    'is_open': False,
    'room_name': 'room_0',
    'name': 'room_0 - door@(7, 7)',
    'obj_id': 'room_0_-_door@(7,_7)_109',
    'location': (7, 7),
    'is_movable': False,
    'carried_by': [],
    'is_traversable': False,
    'class_inheritance': ['Door', 'EnvObject', 'object'],
    'visualization': {'size': 1.0, 'shape': 0, 'colour': '#640000', 'depth': 80, 'opacity': 1.0}
  },

  'room_0_area_110: {
    'room_name': 'room_0', 
    'name': 'room_0_area', 
    'obj_id': 'room_0_area_110', 
    'location': (5, 5), 
    'is_movable': False, 
    'carried_by': [], 
    'is_traversable': True, 
    'class_inheritance': ['AreaTile', 'EnvObject', 'object'],
    'visualization': {'size': 1.0, 'shape': 0, 'colour': '#0008ff', 'depth': 80, 'opacity': 0.1}
  }

  ...


  'room_1_-_wall@(5,_7)_118': {'room_name': 'room_0', name:...}
  ........................
  'Drop_off_0_338', {'drop_zone_nr': 0, 'is_drop_zone': True, 'is_goal_block': False, 'is_collectable': False, 'name': 'Drop off 0', 'obj_id': 'Drop_off_0_338', 'location': (12, 21), 'is_movable': False, 'carried_by': [], 'is_traversable': True, 'class_inheritance': ['AreaTile', 'EnvObject', 'object'], 'visualization': {'size': 1.0, 'shape': 0, 'colour': '#878787', 'depth': 80, 'opacity': 1.0}}
  'Drop_off_0_339', 
  'Drop_off_0_340', 
  'Collect_Block_341', 
  'Collect_Block_342', 
  'Collect_Block_343', {'drop_zone_nr': 0, 'is_drop_zone': False, 'is_goal_block': True, 'is_collectable': False, 'name': 'Collect Block', 'obj_id': 'Collect_Block_343', 'location': (12, 21), 'is_movable': False, 'carried_by': [], 'is_traversable': True, 'class_inheritance': ['CollectableBlock', 'EnvObject', 'object'], 'visualization': {'size': 0.5, 'shape': 1, 'colour': '#0008ff', 'depth': 85, 'opacity': 0.5}}

  'World': {
    'nr_ticks':0, 
    'curr_tick_timestamp':1609927984782732,
    'grid_shape':(24,25),
    'team_members': ['agent_0_in_team_0_344', 'human_0_in_team_0_345'],
    'world_ID': 'world_1', 
    'vis_settings': {'vis_bg_clr': '#C2C2C2', 'vis_bg_img': None}} 
}

The World object is somewhat special. The other objects contain a 'obj_id', 'class_inheritance" field and other fields that make it possible to identify the type of object contain in here.

Blocks

Blocks only appear when the agent is near the block (within block_sense_range, see setup section), you get an additional item like

Block_in_room_6_330:{
  'is_drop_zone': False, 
  'is_goal_block': False, 
  'is_collectable': True, 
  'name': 'Block in room_2', 
  'obj_id': 'Block_in_room_2_318', 
  'location': (20, 5), 
  'is_movable': True, 
  'carried_by': [], 
  'is_traversable': True, 
  'class_inheritance': ['CollectableBlock', 'EnvObject', 'object'],
  'visualization': {'size': 0.5, 'shape': 1, 'colour': '#0dff00', 'depth': 80, 'opacity': 1.0}}

'is_goal_block' is set for the "GhostBlock" objects that are placed in the dropzone. These objects contain also a 'visualize' property with 'colour', 'shape' and 'size' values that must match the block that is to be dropped in this zone.

If you move a block, it keeps the same name eg 'Block_in_room_6_330' even though it is not in the said room anymore.

Room doors

The room door objects can have only one roomid assigned. Therefore you can assume that all rooms connect to the corridor.

State functions

state has some utility functionality, for example

get_all_room_names()	the available room_name properties on the map
get_room_objects('room_1')	gives the areas (tiles) that make up room 1
get_traverse_map()	returns dict {(0,0):False, (0,1):True...} where (X,Y) is the map coordinate and the boolean indicates whether the location can be traversed.

You can create a crude map of the environment with the get_traverse_map() like this

        print("Extracting the map")
        map=state.get_traverse_map()
        width, length = state.get_world_info()['grid_shape']
        for y in range(length):
            for x in range(width):
                if map[(x,y)]:
                    print("*",end="")
                else: 
                    print(" ", end="")
            print()

The object fields

Most fields are undocumented and you will have to figure it out yourself or even reverse engineer using the matrx code. Here are a few:

location	a tuple (x,y) with the x,y location of the object on the grid
isAgent	set when the object contains info about an agent
class_inheritance	This is the chain off MATRX used classes the object inherits from. Meaning that a value of [class_B, class_A, EnvObject, object] means that the dictionary with properties comes from a Class_B instance, which inherits from Class_A, which in turn inherits from MATRX’ base object class EnvObject. Finallt, EnvObject in turn inherits from ‘object’; the base Python class from which everything in Python inherits*. So with the snippet state[“some_object_id”][“class_inheritance”][0]| you get the string name of the class definition of the object with ID “some_object_id”.

Path Planner

There is a simple path planner available in matrx, called Navigator. It takes (x,y) coordinates as input, and it needs a state_tracker as well. You can use it like this

    state_tracker = StateTracker(agent_id=self.agent_id)
    navigator = Navigator(agent_id=self.agent_id, 
            action_set=self.action_set, algorithm=Navigator.A_STAR_ALGORITHM)

To use the navigator you need something like

    navigator.reset_full()
    navigator.add_waypoints([targetloc])

navigator has a is_circular option to create a circular path that loops back to the first waypoint after reaching the last. Check the python documentation for details.

If you are not using reset_full, old path points may remain and the path may not work

Now you can extract the next steps using

  self._state_tracker.update(state)   
  action=navigator.get_move_action(state_tracker)
or

It will take the current agent location as start point and navigate to the waypoint you can add later. The StateTracker contains some processed map so better

FAIK it can only reach reachable places, it will not consider opening doors and refuse planning if the end point can not be reached. Once a door is open, path planning will work

Duration of actions

BW4TAgents can not actively use duration of actions. This section is only here to explain how duration of actions actually works.

In the gridworld/BW4T world, all time is measured in ticks. Actions and messages are all executed at a single tick. These ticks are independent from the wall time clock and thus this world can be slowed down without changing the behaviour. In a typical simulation, around 10 ticks are executed in a second, but this may differ depending on the CPU speed and load.

In grid_world, all actions you can also set 'action_duration'. The minimum is 1, if you set it smaller matrx will change it to 1. This field indicates after how many ticks the action will actually be performed, so 10 means that nothing happens the coming 9 ticks and the action will then be performed in tick 10 from now.

In BW4T agents can NOT change the action_duration, because BW4T uses the action_duration field to control the overall speed of an agent. Your agent will raise an exception if you try to set action_duration.

'duration' works like this: once an agent decides to do an action by returning this from the decide_on_action function, e.g. move north, with an action duration of 5, that agent will be "busy" for 4 ticks, and do the action on the 5th. The agent being busy is indicated with the gears animation in the frontend.

What this means for the agent is that when the agent is busy with an action, the agent can only perceive the world, but not decide on a new action: only filter_observations is called for agents that are busy with an action, and decide_on_action is skipped. In that manner the situation you propose is impossible, as the agent is blocked from deciding on any new actions once it has initiated an action that takes multiple ticks.

The next required block

The BW4T world only accepts the placed blocks if only the correct blocks are placed in the right places in the right order. So it is not acceptable if other blocks are placed on the dropzone as well, nor is placement accepted if the order of placement was incorrect.

Since agents only can see the blocks when they are very close to the blocks, agents can usually not determine the order in which the blocks were placed, thus not whether the placement order is correct. So if all blocks are there and the game is not completed, the only solution then is to re-place all the blocks.

Block Shapes

The blocks have a number of options for their shape

shape number	shape
0	rectangle
1	triangle
2	circle

Communication

A basic communication mechanism is provided to all agents. Use this to send a message:

    msg = Message(content='Delivering block'), from_id='me' )
    self.send_message(msg)

This example message is not too informative but you can extend it to make it more useful. The from_id is a required field but can contain anything. It probably is useful to put the agent ID there. Check the matrx.messages.Message class for more details.

If you do not specify to_id, your message will be broadcasted to all agents, including yourself.

Received messages appear some time later (probably the next tick) in self.received_messages list. The received messages remain in the list only one tick so you need to scan them every tick if you don't want to miss messages. (but see ​issue)

Log files

In the world_1 directory a log file is written for each session. It's in the CSV file format, with ";" as the column separator. Its first line contains the column information. Each row contains the info for one tick.

Column	meaning
done	contains True if the goal has been achieved, else False.
Agent*_msgs	contain the number of messages sent by the agent in the previous tick
Agent*_acts	The action done by the agent in this tick
tick_nr	The tick value for this row

Running the tests

To run the tests using eclipse pyDev,

• Set the PYTHONPATH, add the root of project as source folder
• All test functions must have names test_XXX. Otherwise it don't work

With that, right click on the project and select 'run as / python unit test'.