The document discusses 5 levels of generative AI that could be applied to games and virtual worlds, inspired by levels of autonomous vehicles. It outlines the levels for 4 types of creators: game studios, modders, players, and the game itself. The levels range from no automation to direct creativity from imagination. For each creator type, level 5 represents a state where generative AI is seamlessly integrated to directly spawn creations from ideas or prompts. The document aims to help identify opportunities for generative AI and mark progress in virtual world innovations.

1. Games and Virtual Worlds:
The 5 Levels of Generative AI
Jon Radoff

2. Generative AI for
Games
With recent advances in
generative AI, there has been a
lot of discussion about how it
will be used in games, virtual
worlds and the metaverse.
We need a way to identify the
big opportunities and mark
progress towards some of the
more amazing innovations.
Google Trends: Interest in Generative AI as of January 25, 2023

3. Inspiration from
Autonomous Cars
SAE International developed 5
levels of autonomous features
for cars.
What if we applied a similar
progress to the work of creating
virtual worlds and games?
Source: https://www.sae.org/standards/content/j3016_202104/

4. Games feature many
types of creators
While autonomous cars will do
work for us—in virtual worlds, we
can express ourselves creatively and
affect the product itself.
While much of the initial focus on
generative AI in games has been the
potential to accelerate production
pipelines at game studios—a more
holistic view of games must account
for all of the creators who touch
games. Let’s look at these
categories of creators…

5. Game
Player
Modder
Studio
The developers create the
experience, using engines & tools
that exist outside a particular game
framework.
Examples: games built on Unity and
Unreal or bespoke engines; Elden
Ring, Grand Theft Auto, Call of Duty,
World of Warcraft and most games
that have ever been made.
Creative continuum for games & virtual worlds
Extensions of the experience
created outside the game loop
using external tools.
Examples: Minecraft mods, Terraria
mods, World of Warcraft UI mods,
Neverwinter Nights, Quake, Roblox
games–or old-fashioned hacking
like using a sector-editor.
Some games feature creativity that
takes place inside the game.
Examples: Minecraft; Sim City; Star
Wars Galaxies; social creativity such
as guilds in MMOs; meta-creativity
like Twitch streaming
The game itself creates content
while you play.
Examples: AI Dungeon; Replika;
Hidden Door; but includes older
procedural techniques such as those
in Elder Scrolls games and
Roguelikes (e.g,., Diablo series).
In games, these domains of creativity are messy,
overlapping and growing more emergent

6. Generative to pre-generative
The levels for generative content in games also needs to reflect the background in “pre-generative”
technologies such as automation, 3D engines and procedural content that already accelerated
aspects of game development.
Let’s map these categories of creators against the history and future of generative technology for
games:

7. Generative Levels: Studio-as-creator
LEVEL 0
No Automation
No meaningful automation (but might
kitbash a bit). Creator builds their own
engine & backend.
LEVEL 1
Creator Assistance
Uses 3D engine and/or backend
platform as compositional framework.
Leverages off-the-shelf tools and code.
LEVEL 2
Partial Automation
Non-technical tools for creative inputs.
Tech eliminates previous steps.
Serverless & containers adopted.
LEVEL 3
Conditionally Generative
Generative AI used for some parts of
the pipeline. Human heavily in the
loop, concept/story iteration, code gen.
LEVEL 4
Highly Generative
Total automation of 1+ creative
pipelines, e.g., text prompt to make
completely usable 3D models.
LEVEL 5
Direct from Imagination
Multiple generative pipelines
integrated comprehensively into a
composition..
Creativity happens outside an established game system

8. Generative Levels: Modder-as-creator
LEVEL 0
Hackers Only
The game has no inherent modding.
The way it all began: hackers with old-
fashioned sector editors, disassemblers.
LEVEL 1
Game-provided Tools
The game comes with tools such as
map-editors or accessible interfaces to
apply off-the-shelf creative tools.
LEVEL 2
Scripting Interfaces
Game provides an API to build mods, or
provides an embedded script interface
for creating mods.
LEVEL 3
Conditionally Generative
Generative AI used for some parts of
the pipeline (concept art, story, etc.)
with a human heavily in the loop.
LEVEL 4
Highly Generative
Integrated generative tooling, e.g.,
tools like map editors invoke
generative AI to accelerate creation.
LEVEL 5
Direct from Imagination
Complete mod is spawned from an
idea. Iterative ”sculpting” of the mod
through prompts used for refinement.
Creativity happens within a game system, but outside the core game loop

9. Generative Levels: Player-as-creator
LEVEL 0
Limited Creativity
The creativity is limited to tactics and
strategy pertaining to mastering the
game itself.
LEVEL 1
Player-as-Content
The player alters a game by affecting
social systems & economy, but doesn’t
meaningfully impact content.
LEVEL 2
Worldbuilding
The player’s actions affect persistent
content of the world, like living
narratives and content.
LEVEL 3
Emergent Narrative
The player impacts the shape of the
world itself, such as how maps and
structures live in a persistent world.
LEVEL 4
Integrated Generativity
Things formerly the domain of
modding become part of the game
loop, along with in-loop generative AI.
LEVEL 5
Direct from Imagination
If players can dream of it, they speak it
into existence; game system fits it to
rules in an appropriate manner.
Players express themselves within the core game loop

10. Generative Levels: Game-as-creator
LEVEL 0
Static Games
The game doesn’t change in significant
ways between gameplay experiences.
Set level progression, etc.
LEVEL 1
Procedural Content
Rule-based procedural content such as
unique maps during play. May also
include basic AI opponents.
LEVEL 2
Live Generative Content
Generative content in-the-loop utilizes
rule-based systems (such as RLHF) to
maintain consistency & structure.
LEVEL 3
Conditionally Generative
Experience includes content generated
“live” while playing: text, images, music
or NLP-aware characters.
LEVEL 4
Highly Generative
”Game as Dungeon Master,” game
understands what a player likes most
and personalizes/adapts experience.
LEVEL 5
Full Computational Creativity
Explores the search space of human
fun: like a game-designer friend who
comes up with new systems.
The game loop itself contains generative experiences

11. What level of generative
technology do you aspire to in your
own games and virtual worlds?
Jon Radoff | Metavert
https://twitter.com/jradoff
https://linkedin.com/in/jonradoff
This Composition is licensed under
Creative Commons Attribution 4.0