How the engine works

Grand Theft Auto IV runs on RAGE, Rockstar's own engine. This page shows how it is put together, from what the project has found so far in the game's program file and data.

Four lines of work run at the same time

A frame is one finished picture; the game makes a new one many times a second. A thread is one worker inside the program. Each line below is one worker, or a small group of them, and each stop is a job done on that line. Press any stop to see what we found there.

Game thread

the main thread

Moves the world forward one step.

The draw list goes to the render thread. We infer that the game thread then starts on the next frame; the loop that would prove it is not readable yet.

Render thread

1 thread

Turns the draw list into a picture.

Audio

4 threads, drawn as one line

Keeps its own schedule, apart from the picture.

Streaming

thread count not known yet

Loads the city around you as you move.

Thread counts are measured where a number is shown. The order of stops along a line is our reading of the evidence: the main loop itself sits in the part of the program that cannot be read yet, described at the end of this page. The moving markers only show that the lines run at once; they imply no timings.

Nearly a third of the functions are about people

The program holds about 43,000 functions that we can read. Sorted by what they work on, the largest group by far belongs to pedestrians: how they decide, react, walk and fight.

  1. People: AI tasks and events13,286
  2. Timing, locking, start-up4,419
  3. Streaming and loading4,416
  4. Files and memory3,261
  5. Physics and collision3,240
  6. Input, menus and HUD2,974
  7. Network and online2,893
  8. Script machine and natives2,863
  9. Vehicles2,586
  10. Audio2,370
  11. World and object pools1,620
  12. Animation and Euphoria1,276

This counts functions, not bytes of code. Each function is assigned to the nearest subsystem whose code could be identified, so the split is approximate; 25 could not be placed. The bars were counted on the first inventory of 45,229 entries. A later repair removed 4,540 that were not functions and added 2,467 that had been missed, and later checks removed 332 more, leaving 42,824. That changes the totals but not the picture. The counts include the libraries built into the program and leave out the megabyte that cannot be read yet. There is no rendering group because the survey behind these labels did not have one: rendering functions are spread across the groups above, and much of the low-level graphics code is in the unreadable megabyte.

More than half of the classes are leaderboards

A class is a blueprint for one kind of object. The program declares 3,727 of them. 1,927 are leaderboard classes, nearly all of them four templates stamped out once for every one of 481 multiplayer leaderboards. The largest family that does real work is the AI's 334 task classes.

  • Leaderboard classes, 1,927
  • AI tasks, 334
  • Everything else, 1,466

A pedestrian thinks in layers

Nobody in Liberty City runs one big behaviour routine. Each person holds a goal, the goal picks a smaller step, and the step picks one action. When something happens nearby, an event arrives and can replace any of the three layers.

Events A gunshot, damage, a shout. 82 kinds. An event can replace any layer.
  1. Goal A complex task. It chooses a step and watches it. 197 kinds of complex task. In plain words: get to the other side of the street.
  2. picks
  3. Step Another complex task. Steps can nest several deep. Walk to the kerb.
  4. picks
  5. Action A simple task. It does one thing for a few frames, then reports back. About 130 kinds. Take one stride.
  6. drives
  7. The body moves Movement and animation, or Euphoria if the body goes limp.

The counts are measured from the program's class records. The street-crossing example is ours, to show the idea; it is not taken from the game's task names.

  • About 192 bytes is the typical size of a complex task. Tasks live in fixed pools, not in general-purpose memory.
  • 39 movement tasks carry a second interface so the movement system can drive them directly.
  • 18 Euphoria behaviours take over when a body turns ragdoll.
  • We found no "wanted level" class in the readable code. Police response appears to be driven by data files and scripts.

The story runs on a small stack machine

Missions are not built into the program. They ship as 941 script files, run by an interpreter: a loop that reads one instruction at a time and carries it out. It knows 78 instructions and keeps its working numbers on a stack, like a pile of plates. Scripts cannot touch the engine directly. They call named commands, called natives, and the engine does the work.

Program

  1. PUSH 5
  2. PUSH 7
  3. ADD

Stack

Press Step to run the first instruction.

This three-instruction program was the first thing the project ran through the game's original interpreter code. It answered 12. A real script does the same thing with a native in place of ADD: push the arguments, call the native, read the answer off the stack.

  • 14,499,370 instructions across all the shipped scripts.
  • 717,139 calls from scripts into the engine.
  • 3,034 natives registered, every one now named.
  • IS_CHAR_INJURED is the most-called native, followed by PRINTSTRING and IS_CHAR_DEAD.
  • Dividing by zero does not crash a script: the result is the number you started with.
  • 46 debug natives were stripped from the release build and now all lead to one empty function.

The city is 23.9 GB of nested containers

Almost everything sits inside archives. Inside an archive, each model, texture set or collision mesh is a compressed resource, laid out so it can be copied into memory and used without further parsing.

ArchiveA table of contents and thousands of files. The map archives are encrypted with a key held in the program file.
ResourceOne compressed file, split into a part for main memory and a part for the graphics card.
ObjectA model, texture set, collision mesh or navigation mesh, ready to use once the addresses inside it are corrected.
  • 26,713 model files.
  • 15.5 million collision polygons in 9,102 shapes.
  • 10,839 navigation meshes holding 5.65 million polygons.
  • 95,137 road nodes joined by 159,902 links.
  • 10,134 shader programs, all compiled ahead of time.
  • 7 languages of text.

The project's Rust readers open every one of these file types with no failures. The model reader is checked against a sample of 112 files so far.

The clock counts in whole milliseconds

The engine reads a millisecond clock from more than 80 places and a high-resolution timer from one. We found no fixed step in the physics: the step follows the length of the frame. At 30 frames per second a millisecond is a fine ruler. At several hundred it is not.

One frame, start to end. Each tick is one millisecond.

The fraction of a millisecond at the end of the frame, which a whole-millisecond clock cannot see.

Which of the two clocks paces the frame is in the code we cannot read yet, so this ruler illustrates a risk. It is not a measured cause. What is public record: players report that handling, ragdolls, helicopter rotors and ladders misbehave above 60 frames per second.

One megabyte is still dark

The first megabyte of the game's code is encrypted in the file on disk and is only decrypted when the game starts. That is about 9% of the code, and it holds some of the most central parts.

  • Encrypted on disk: 250 pages, 9%
  • Readable: 2,424 pages

Inside the dark part, as far as the readable code's references show:

  • The main loop and the program's real starting point.
  • The memory allocator that every object is created through.
  • 98.8% of the virtual functions of the multiplayer replication classes.
  • 83% of the virtual functions of the core classes: file devices, memory, parsing.
  • 92 to 98% of those of the low-level graphics classes.
  • 63% of those of the AI's event classes.

The plan is to read that megabyte from the owner's running copy of the game, then repeat the analysis on it. Every number on this page comes from an entry in the devlog, where the method and the open questions are written down.