Tools

The programs and scripts the project runs, what each is for, who can run it and the command. Every one is in the repository; the commands are run from its root.

Who can run what

  • Anyone with the repository Needs only a checkout, Python 3 and, for the Rust tools, the Rust toolchain.
  • Needs your own copy of the game Reads files from a copy of Grand Theft Auto IV that you bought, on Steam, from an authorised key seller or from a retailer. Nothing from the game is in the repository or in the tools, and what they produce from your files must not be committed or shared.
  • Windows Builds or runs a 32-bit Windows program.
  • The project's workstation Reads the agent lanes' working files, which are never published.

Reading the game's files

  • lf-inspect

    One command-line tool for every file format the project reads: archives, resources, textures, models, collision, navigation, text, scripts, shaders, audio, animation, cutscenes and more. It prints summaries, listings and dumps, and measures how fast each reader parses a file. It replaces the example programs the format crates used to carry. Every command only reads; where an archive needs a key, the key is found in your copy's executable at run time and kept in memory only.

    Needs your own copy of the game Anyone with the repository for the synthetic benchmark, which makes its own inputs.

    • cargo run --release -p lf-inspect -- formats
    • cargo run --release -p lf-inspect -- <format> [<command>] <file>
    • cargo run --release -p lf-inspect -- bench synthetic [--iters N]

    Source: crates/tools/lf-inspect

  • lf-viewer

    Converts a model and its textures from your own game files into standard files any viewer opens: glTF 2.0 for meshes and PNG for textures. It is a command-line converter, not a window. It reads only the paths you pass and writes only into the output folder you name, never overwriting without --force.

    Needs your own copy of the game

    • cargo run --release -p lf-viewer -- model <MODEL> --out <DIR> [--textures <FILE.wtd>]
    • cargo run --release -p lf-viewer -- texture <TEXTURES.wtd> --out <DIR>
    • cargo run --release -p lf-viewer -- --help

    Source: crates/tools/lf-viewer

  • Shader converter

    An example program of lf-dxso-spirv, the translator from the game's Direct3D 9 shader bytecode to SPIR-V for Vulkan. It translates every program in one compiled shader container and writes one .spv file per program into a folder that must already exist, printing each program's stage, size and interface.

    Needs your own copy of the game

    • cargo run --release -p lf-dxso-spirv --example lf_dxso_convert -- <file.fxc> <output-folder>

    Source: crates/lf-dxso-spirv

Checking rewrites

How these fit together is explained on the quality page.

  • The checker (checker2.py)

    Decides whether a Rust rewrite of one original function behaves the same as the original. It runs both from bit-identical starting state over many generated inputs and compares the return value, memory writes, the stack pointer, outgoing calls and faults. One contract per function says what to generate and what to compare. A 32-bit worker process runs the trials; this script drives it and writes the verdicts.

    Needs your own copy of the game Windows The --selftest checks of the driver itself run anywhere.

    • cargo build --release --target i686-pc-windows-msvc -p lf-checker-worker
    • RUSTFLAGS="-C panic=abort" cargo build --release --target i686-pc-windows-msvc -p lf-checker-proofs
    • python scripts/checker/checker2.py CONTRACT [CONTRACT ...]
    • python scripts/checker/checker2.py --smoke
    • python scripts/checker/checker2.py --selftest

    Source: scripts/checker/checker2.py

  • validate_contracts.py

    Checks contracts before they run. The checker ignores keys it does not know, so a misspelt setting would silently drop an observation; this makes such mistakes loud. It reports errors (do not run), warnings (narrowings: what the proof will not cover, by name) and notes.

    Anyone with the repository

    • python3 scripts/checker/validate_contracts.py
    • python3 scripts/checker/validate_contracts.py PATH [PATH ...]
    • python3 scripts/checker/validate_contracts.py --json ...
    • python3 scripts/checker/validate_contracts.py --quiet ...

    Source: scripts/checker/validate_contracts.py

  • mutate_rewrite.py

    Writes deliberately wrong versions of a rewrite, each with one change (a flipped comparison, a constant off by one, a dropped write, swapped call arguments and more), to find contracts that cannot see a change. A wrong version that passes the checker marks a blind spot.

    Anyone with the repository to write them; running them through the checker needs your own copy of the game

    • python3 scripts/checker/mutate_rewrite.py FILE [FILE ...] --out DIR [-n 8] [--seed 0]
    • python3 scripts/checker/mutate_rewrite.py FILE ... --dry-run [--json]

    Source: scripts/checker/mutate_rewrite.py

Keeping the tree honest

  • The ledger (ledger.py)

    Checks the tracked rewrite tree against itself and against the published counts: an index entry without its file, a file no index names, an address listed twice or in both halves, a file name that disagrees with its address. It also validates the optional proof records. CI runs it with --check.

    Anyone with the repository

    • python scripts/coordinator/ledger.py
    • python scripts/coordinator/ledger.py --json
    • python scripts/coordinator/ledger.py --check

    Source: scripts/coordinator/ledger.py

  • lint_rewrites.py

    Pattern checks over every verified rewrite, for what the checker cannot see: arithmetic that panics in a debug build, plain reads at odd offsets, an export that is not the code the checker ran, a file that says its own proof is partial. Each finding says whether the text alone proves it or a heuristic suggests it.

    Anyone with the repository

    • python scripts/coordinator/lint_rewrites.py
    • python scripts/coordinator/lint_rewrites.py --json
    • python scripts/coordinator/lint_rewrites.py --write PATH

    Source: scripts/coordinator/lint_rewrites.py

  • check_rewrites_build.py

    Answers whether every verified rewrite compiles from the repository alone. Each lane compiled its rewrites with its own helpers; this type-checks every file against the shared runtime only, for the 32-bit Windows target, dropping failures and repeating until the rest compiles. No linker is needed, so it runs on any machine with that Rust target.

    Anyone with the repository with the i686-pc-windows-msvc Rust target installed.

    • python scripts/ci/check_rewrites_build.py [--out PATH] [--edition 2021|2024]

    Source: scripts/ci/check_rewrites_build.py

  • issue_log.py

    Collects every known problem with a verified rewrite into one file: the pattern checks, the build check's failures and reading reviews. Every issue keeps its source and status, so a reviewer's verdict survives the next regeneration. The quality page is drawn from it.

    Anyone with the repository

    • python scripts/coordinator/issue_log.py [--build PATH] [--review PATH] [--out PATH]

    Source: scripts/coordinator/issue_log.py · the issue log

Building for the game

  • assemble.py

    Assembles the verified rewrites into one injectable 32-bit library for the in-game test. It cuts them into shard crates of a few hundred each, compiles each shard against the shared runtime alone, leaves out what does not compile with the reason recorded, and writes a manifest of what went in. Generated code goes under .artifacts/ and is never committed.

    Anyone with the repository for the type-check; Windows for --build, which needs the Microsoft linker.

    • python scripts/assemble/assemble.py
    • python scripts/assemble/assemble.py --callee-map FILE
    • python scripts/assemble/assemble.py --build-shards
    • python scripts/assemble/assemble.py --build

    Source: scripts/assemble/assemble.py

Running the project

  • The dashboard

    A local page for watching the agent lanes without a model in the loop: every lane, its log and results, and the machine's load. It reads what is already on disk and starts nothing. It listens on the machine itself only, because lane logs hold paths and addresses that are not for publication.

    The project's workstation

    • python scripts/dashboard/server.py [port]

    Source: scripts/dashboard/server.py (default port 8772)

  • The site's generators

    The data behind this site. devlog_db.py stores devlog entries in the database the devlog page searches; devlog_feed.py writes the devlog's Atom feed; history.py writes the progress-over-time series from the git history of the progress file; quality.py writes the quality page's counts; social.py draws the link-preview image. The feed and the quality data are checked against the publication rules before they are written; a lane's devlog entry is checked by integrate_devlog.py on its way in.

    Anyone with the repository

    • python scripts/devlog_db.py put <fragment.html>
    • python scripts/site/devlog_feed.py
    • python scripts/site/history.py
    • python scripts/site/quality.py
    • python scripts/site/social.py

    Source: scripts/devlog_db.py · scripts/site

  • The publication check

    Fails if a tracked file breaks the repository's publication rules: anything from the game folder or the scratch area tracked, game or binary files, a path on someone's machine or an account name, inline assembly in a rewrite, or a file over the size limit. The repository is public and every commit is pushed, so CI runs it on every push.

    Anyone with the repository

    • python scripts/ci/check_publication.py

    Source: scripts/ci/check_publication.py

Glossary

Lane
One AI agent working on one assignment: a batch of functions to rewrite or name, or a research question. A lane writes only in its own scratch folder and never commits; a script hands it its work.
Contract
The file that tells the checker how to test one function: how to call it, which inputs to generate, what each function it calls should answer, and what to compare.
Checker
The program that runs an original game function and its Rust rewrite from identical starting state, many times, and compares everything a caller could observe. Its verdict, not an agent's opinion, decides whether a rewrite passes.
Wrong version
A copy of a rewrite with one deliberate mistake, run through the same contract. It must fail; if it passes, the contract cannot see that kind of change.
Verified
Accepted by the checker under version 2 or later. The overview's verified count includes game code only.
Rewritten
Has a Rust implementation. Every verified function is also counted as rewritten.
Named
Has a meaningful name instead of a placeholder.
Lift
Restating a verified 32-bit rewrite as portable, safe Rust: no addresses, no numbered callee slots, no global state, and the same results on a 64-bit machine. It is the step from rewrites that run inside the original game towards a game that runs without it.
Shard
One crate of a few hundred verified rewrites, cut in address order. The library for the in-game test is built shard by shard so that no single build is too large for the machine.
Callee slot
The numbered entry through which a rewrite calls another function. In the checker every slot is a recording stub; in the game each slot must point at the real function, which a callee map supplies.
Tick
The coordinator's five-minute cycle: it brings in what finished lanes produced, updates the progress data behind this site, commits and pushes.