The competition software (environments, trainers, and some agents) are based on the RL-Viz project directed by Brian Tanner at the University of Alberta.

 RL-Viz has an official homepage, but not much is there yet because we haven't done an official release of the software.  Look for that in the spring of 2008.
http://rl-viz.googlecode.com

RL-Viz is a platform for developing, debugging, benchmarking, teaching, and demonstrating reinforcement-learning methods.  RL-Viz is currently a Java library for dynamically connect reinforcement learning agents, environments, and experiments.  RL-Viz is a protocol and software library layered over RL-Glue.

Using RL-Viz, the reinforcement learning community can create agents and environments whose parameters can be changed on the fly.  Creating an ad-hoc experiment is as easy as choosing an environment from one list, an agent from another list, and then clickign "run experiment".  RL-Viz allows custom visualization tools to be written for agents and environments to view dynamics, value functions, scores, etc.  It makes debugging RL environments and agents much easier, and allows for very cool visual demos.

 The current RL-Competition software uses only a small part of what RL-Viz has to offer.  Some of the cool aspects of RL-Viz will make it into upcoming releases for the RL-Competition software. 

Tetris and RL-Viz