RoboCup simulated soccer is a well-understood domain, as it has been the basis of multiple international competitions and research challenges. The multiagent domain incorporates noisy sensors and actuators, as well as enforcing a hidden state so that agents only have a partial world view at any given time. Since late 2002, the Keepaway task has been part of the official release of the open source RoboCup Soccer Server used at RoboCup. Agents in the simulator receive visual perceptions every 150 msec indicating the relative distance and angle to visible objects in the world, such as the ball and other agents. They may execute a primitive, parameterized action such as turn(angle), dash(power), or kick(power,angle) every 100 msec. Thus the agents must sense and act asynchronously. Random noise is injected into all sensations and actions. Individual agents must be controlled by separate processes, with no inter-agent communication permitted other than via the simulator itself, which enforces communication bandwidth and range constraints.

When started in a special mode, the simulator enforces the rules of the Keepaway task, instead of the rules of full soccer. In Keepaway one team - the keepers - attempts to maintain possession of the ball within a limited region while another team - the takers - attempts to steal the ball or force it out of bounds, ending an episode. Whenever the takers take possession or the ball leaves the region, the episode ends and the players are reset for another episode (with the keepers being given possession of the ball again). Standard parameters of the task include the size of the region, the number of keepers, and the number of takers. Other parameters such as player speed, player kick speed, player vision capabilities, sensor noise, and actuator noise, are all adjustable.

The primary difficulties when learning Keepaway are the 13-dimensional continuous state space, noisy sensors, and noisy actuators. However, methods including temporal difference, genetic algorithms, and approximate policy iteration have all been used to learn successful keeper policies. Competitors may wish to refer to previously published work on Keepaway to see what has been previously successful, but we anticipate many novel approaches will also be able to learn competitively in this domain. More information about Keepaway can be found here.

The competition domain is based on the publicly available benchmark task but will be altered between training, proving, and testing.  See the Rules Page for more information about the altered evaluation paradigm.

Technical Details

Observation Space: 13 dimensional, countinuous valued (3 keepers and 2 takers)

• dist(K1 , C ); dist(K2 , C ); dist(K3 , C );
• dist(T1 , C ); dist(T2 , C );
• dist(K1 , K2 ); dist(K1 , K3 );
• dist(K1 , T1 ); dist(K1 , T2 );
• Min(dist(K2 , T1 ), dist(K2 , T2 ));
• Min(dist(K3 , T1 ), dist(K3 , T2 ));
• Min(ang(K2 , K1 , T1 ), ang(K2 , K1 , T2 ));
• Min(ang(K3 , K1 , T1 ), ang(K3 , K1 , T2 ));

where K1, K2 and K3 are the keepers and T1 and T2 are the takers and C is the center of the playing region. dist(a, b) is the distance between a and b and ang(a, b, c) is the angle between a and c with vertex at b.

Action Space: 1 dimensional, discrete valued

1. pass to nearest keeper
2. pass to other keeper
3. hold ball
   

Rewards: positive function the number of time steps the ball is "kept"

 
Note 1: Additional the installation documentation for the Keep Away domain can be found here .

 
Note 2: the competition software will provide your agent with a task specification string that describes the basic inputs and outputs of the particular problem instance your agent is facing. For the competition, the ranges provided in task specification may not be tight; they provide a rough approximation of the actual observation and action ranges. More documentation of the the task specification string can be found here .