This presentation describes the design of a bot for the first person shooter Unreal Tournament— 2004, which behaves as a human expert player in 1 vs. 1 death matches. This has been implemented modelling the actions (and tricks) of this player, using a state-based AI, and supplemented by a database for ‘learning’ the arena. The expert bot yields excellent results, beating the game default bots in the hardest difficulty, and even being a very hard opponent for the human players (including our expert). The AI of this bot is then improved by means of three different approaches of evolutionary algorithms, optimizing a wide set of parameters (weights and probabilities) which the expert bot considers when playing. The result of this process yields an even better rival; however the noisy nature of the fitness function (due to the pseudostochasticity of the battles) makes the evolution slower than usual.

1. Designing and Evolving an
Unreal Tournament 2004
Expert Bot
A.M. Mora, F. Aisa, R. Caballero, P. García-Sánchez, J.J. Merelo, P.A. Castillo, R. Lara-Cabrera
International Work-Conference on Artificial Neural Networks

2. INDEX
• Unreal (game, environment)
• Pogamut tool
• Unreal Expert Bot (objectives, features)
• Evolutionary Algorithms
• Unreal Expert Bot Evolution (description,
approaches, results)
• E-BOT vs GE-BOT (results, demo)
• Conclusions

3. Environment description
UNREAL
Unreal is a first person shooter (FPS).
Famous due to the excelent AI of the enemies (bots), which makes it an
amazing multiplayer game. Unreal Tournament series is widely extended.
It offers an editor (UnrealEd) which lets us change almost anything in
the game even the behaviour of the bots. It uses the language
UnrealScript.

4. General description
POGAMUT
A java middleware for Unreal Tournament series games and
Defcon games.
The architecture is as follows:
It is possible to interact with the game from a java program,
getting higher independence (avoiding Unrealscript restrictions)
and increasing the possibilities (java libraries).
On the contrary, the structures, classes, functions and
workflows defined in the Unreal engine, cannot be accessed, nor
used.

5. Objectives
UNREAL EXPERT BOT
• Create an autonomous agent for playing Unreal
Tournament deathmatch championship.
• Considering the constraints of this competition:
- 1 vs 1 matches
- Small arenas
- Weapons are not respawned
- Some forbidden items (U-Damage, for instance)
- 15 minutes per match instead of a number of frags (kills)
• Human-like behaviour is desired.
• Modelling Expert player knowledge (and tricks).
- High control in timing (items respawn time)
- Deep knowledge about weapons and their advantages and
disadvantages
- Deep knowledge about items

6. Features
UNREAL EXPERT BOT
• Defined by means of a Finite State Machine based AI with two
state levels.
• Translated into a set of rules which determine its behaviour.
• Database which models the bot’s memory, since it is uploaded
with data about locations of items and weapons in the map.

7. Features
UNREAL EXPERT BOT
• Defined by means of a Finite State Machine based AI with two
state levels.
• Translated into a set of rules which determine its behaviour.
• Database which models the bot’s memory, since it is uploaded
with data about locations of items and weapons in the map.

8. Bot performance
UNREAL EXPERT BOT
• Expert Bot (E-Bot) outperformed the standard bots in the game
(considering the number of frags), even in the maximum
difficulty level.
• This difficulty level is quite hard for a medium level
player.
• E-Bot is hard to beat for humans, even for the expert.
• Medium level players usually lose against it.

9. Evolutionary Algorithms
EXPERT BOT EVOLUTION
by Johann Dréo
i -> initial
population
f -> evaluation
function (fitness)
? -> stop condition
Se -> selection
Cr -> crossover
Mu -> mutation
Re -> replacement

10. Evolutionary Process in Unreal game
EXPERT BOT EVOLUTION
GE-BOT
Expert Bot based in a
Genetic Algorithm
Evolutionary
process
population
FITNESS EVALUATION
• Analyze Expert bot’s FSM
• Identify parameters
• Optimize them
Expert
Bot’s
AI

11. Approaches
EXPERT BOT EVOLUTION
• Generic Fitness
Just considers frags/deads
and damage produced/received
• Generational scheme
• 4-elitism
• Complex Fitness
- considers frags/deads
- damage produced/received
- time using the best or more
versatile weapons: Lightning
Gun and Shock Rifle
- getting the best items: Shield
and Super Shield
• Stationary scheme
• Chromosome 143
• Uniform Crossover
• Random mutation
• 4 Random individuals
• Chromosome 26

12. Approach 1: Chromosome 143 - Generic Fitness
EXPERT BOT EVOLUTION
• Generic Fitness
Just considers frags/deads
and damage produced/received
• Generational scheme
• 4-elitism
• Chromosome 143
• Uniform Crossover
• Random mutation
• 4 Random individuals

13. Approach 1. Results
EXPERT BOT EVOLUTION
• 30 generations
• 30 individuals
• 1 evaluation (left)
• 3 evaluations (right)
in order to avoid the
noisy nature of the
fitness function
• 15 minutes per
evaluation
• 10 days per run (left)
• One month (right)
• Lightly improvement
tendency
• Too many oscillations,
i.e. noise
• 143 genes are too much

14. EXPERT BOT EVOLUTION
• Generic Fitness
Just considers frags/deads
and damage produced/received
• Generational scheme
• 4-elitism
• Uniform Crossover
• Random mutation
• 4 Random individuals
• Chromosome 26
Approach 2: Chromosome 26 - Generic Fitness

15. Approach 2. Results
EXPERT BOT EVOLUTION
• 50 generations
• 30 individuals
• 5 minutes per
evaluation
• Results of 2 different
runs
• 5 days per run
• Again lightly
improvement tendency
• Too much noise
• Too much diversity

16. EXPERT BOT EVOLUTION
• Complex Fitness
- considers frags/deads
- damage produced/received
- time using the best or more
versatile weapons: Lightning
Gun and Shock Rifle
- getting the best items: Shield
and Super Shield
• Stationary scheme• Uniform Crossover
• Random mutation
• 4 Random individuals
• Chromosome 26
Approach 3: Chromosome 26 - Complex Fitness

17. Approach 3. Results
EXPERT BOT EVOLUTION
• 40 generations
• 30 individuals
• 5 minutes per
evaluation
• Stationary scheme to
increase the
exploitation factor
• Results of 2 different
runs
• 5 days per run
• Quite good fitness
tendency
• Noise still remains,
but in a lower factor

18. Numerical results
E-BOT vs GE-BOT
• Expert Bot (E-Bot) and the best Genetic Expert Bots (GE-BOT)
have been fighting in four battles (in two maps).
• The average results of these matches are:
• The approach with 143 genes per chromosome is defeated
• GE-Bot with 26 genes outperforms E-Bot.
• The approach with the complex fitness function gets the best
results. Due to its lower noisy factor, and the higher
exploitation component.

19. DEMO
E-BOT vs GE-BOT
http://www.youtube.com/watch?v=ktcXHZ-nAfw

20. CONCLUSIONS
• We have designed a human-like Expert Bot (E-Bot) which
outperforms the standard Unreal Tournament 2K4 bots in the
hardest difficulty.
• It is also a hard rival against human players.
• We have tested three different approaches for improving this
bot by means of Genetic Algorithms.
• Too long chromosomes population performs worse than small
length one.
• These algorithms are affected by a high noisy factor
regarding the generic (and easier) fitness function.
• We have defined a complex fitness function which performs
better, with a softer noisy effect.
• The bots obtained after evolution outperform the E-Bot.

21. END
THE
Questions?!?!
Contact: amorag@geneura.ugr.es
Source Code: https://github.com/franaisa/ExpertAgent