Codex vs Claude Code: Real-Repo Benchmark

Codex vs Claude Code comparisons go wrong when the test is one prompt, one toy repo, and one subjective verdict. Strong coding agents fail in different places: one may plan better, one may edit faster, one may recover from test failures better, and one may produce a cleaner review narrative. Your benchmark should make those differences visible.

• Use the same real repository for both agents.
• Pin the task prompt, branch name, dependency state, and timebox.
• Force both agents to run the same install, lint, test, and browser checks.
• Score the diff, not the chat transcript.
• Track wall-clock time, tokens or subscription usage, retries, and human intervention.
• Run the benchmark on a persistent host so caches, browser state, logs, and task artifacts survive between runs.

Start with a repo that already has some pain: real dependencies, flaky setup, old files, incomplete tests, browser UI, and at least one architectural decision that cannot be solved by search and replace. Then create a task set that exercises different muscles.

A benchmark is useless if one agent gets a warm repo and the other gets a broken checkout. Use the same host class, same branch reset point, same package manager cache policy, same browser profile, same terminal permissions, and same review rules.

1. Create one clean baseline branch and tag it before the first run.
2. Record CPU, memory, disk, OS version, Node/Bun/Python versions, and package-manager versions.
3. Install dependencies once, then decide whether both agents get warm caches or both get cold caches.
4. Create one browser profile for UI tests and reset it between runs when auth state matters.
5. Use the same secrets policy: no production credentials, no payment flows, no account-admin actions.
6. Save every prompt, terminal command, test result, screenshot, and final diff in a run folder.

benchmark-runs/
  2026-05-20-codex-bugfix/
    prompt.txt
    environment.txt
    transcript-notes.md
    commands.log
    test-results.txt
    screenshots/
    final.patch
    scorecard.md
  2026-05-20-claude-code-bugfix/
    ...

Do not score only whether the agent finished. Score the review burden it left behind. A coding agent that lands an 80 percent patch with a chaotic diff may be slower than the one that asks two good questions and ships a narrower change.

The protocol should be boring enough that another engineer can repeat it. This also makes the post stronger: readers trust benchmarks when they can see how you controlled for the environment.

For each task:
1. Reset to the baseline branch.
2. Start a fresh run folder.
3. Paste the exact task prompt.
4. Let the agent inspect the repo before editing.
5. Allow edits, commands, and browser checks within the stated policy.
6. Stop at the timebox or when the agent says the work is complete.
7. Save git diff, command log, screenshots, and the agent's final note.
8. Run the human scorecard before reading the other agent's result.

You may end up using both. One agent can draft the migration plan, another can execute the edit, and a third pass can review for regressions. The benchmark tells you where each one deserves the keyboard.

Hyperbox is not the agent in this comparison. It is the machine the agents run on. That matters when the benchmark needs the same repo, tools, browser sessions, GUI permissions, and logs to stay available while you switch agents or review from another device.

• Run Codex and Claude Code against the same persistent checkout.
• Keep browser and desktop state alive after your laptop closes.
• Capture screenshots and logs from the machine that actually executed the work.
• Use SSH and VNC for terminal-first and desktop-first verification.
• Repeat the benchmark later without rebuilding the entire environment.

Which is better, Codex or Claude Code?

The honest answer depends on the task. Benchmark bug fixes, UI work, refactors, migrations, and dirty-repo handoffs separately, then score correctness, diff quality, verification, recovery, human load, and cost.

What makes a coding-agent benchmark fair?

Use the same repo, baseline branch, prompt, task scope, test commands, browser checks, timebox, host environment, and scoring rubric. Otherwise you are measuring setup differences as much as agent quality.

Why run the benchmark on a persistent Mac?

A persistent Mac keeps dependencies, browser state, GUI permissions, logs, screenshots, and branches available across runs, which makes the benchmark closer to real agent work.