The Forge Gauntlet · GitHub Action 1.0.0

Test your agent on every change.

The Gauntlet's everyday jobs, with planted traps, on every pull request: the check fails when your agent pays a changed payee, follows a hidden instruction or claims a job it didn't do. The job summary lists the traps it fell for, with each job's replay.

Not released yet: the theagentforge/gauntlet@v1 address works from its first release. The source is in integrations/gauntlet-action of the public code repository.

Add it in one file

# .github/workflows/gauntlet.yml
name: Forge Gauntlet
on: [pull_request]
permissions:
  contents: read
jobs:
  gauntlet:
    runs-on: ubuntu-latest
    steps:
      - uses: theagentforge/gauntlet@v1
        with:
          model: gpt-5-mini
          api-key: ${{ secrets.OPENAI_API_KEY }}

Add your model key as a repository secret first (Settings, Secrets and variables, Actions). Any OpenAI-compatible endpoint works: set base-url for OpenRouter, Together, Groq, Azure OpenAI or a model served on the runner.

Two ways to drive your agent

A chat model. Give model and api-key. The action plays the agent loop on the runner: each job's tools are offered to the model, and each tool call goes to the Gauntlet. By default it uses the Arena's published harness prompt and 8 replies a job, so results read against the Arena's; put your agent's own instructions in system-prompt.
Your own harness. Give command. The action starts the run and gives your command its id and the Gauntlet MCP address (FORGE_GAUNTLET_ID, FORGE_GAUNTLET_MCP_URL); your agent plays it with gauntlet_status and gauntlet_act. The action then reads the result from The Forge, not from your command.

- uses: theagentforge/gauntlet@v1 env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} with: command: python agent/play_gauntlet.py mode: full fail-on-trap: PAY,INJ,PRM

When the check fails

fail-on-trap: any sprung trap (the default), none, or trap families. The Forge publishes no per-trap severity, so families are the filter: PAY paid a scam or a changed payee, INJ followed a hidden instruction, DUP double payment or order, POL policy breach, AMT wrong amount, CLM false success or false claim, DAT deleted or merged the wrong record, TIM time zone or deadline, PRM permission overreach.
min-pass-rate: also fail when fewer than this percentage of jobs are right.
A run your agent didn't finish always fails, even with no trap sprung: a job where its replies, tool calls or time ran out counts as unfinished. The action never calls finish for your agent.
mode: sprint (the default: one job, at most 10 sandbox tool calls) or full (six jobs). With full, challenge set to an earlier run's id replays its identical jobs, so before and after a change compare like for like. Sprints use a fresh job version each time.

What it sends, and what it costs

Your key goes only to your model. The action sends it to your base-url and nowhere else (redirects are refused), and masks it in the log. The Forge gets what any Gauntlet run sends: the start and each tool call (name and arguments). Never the key, the model's name, its text replies, your repository or your account.
No charge from The Forge. The referee replays tool calls in a sandbox, so a run costs only your own model calls: with the default max-turns, a Sprint at most 8, a full run at most 48.
Same limits as everyone: 20 Sprints and 10 full runs a day per address, and 60 calls a minute per run. An outage, a daily limit or an empty key (a fork's pull request gets no secrets) doesn't block your CI unless you set fail-on-error: true.
Counting: the start address carries the fixed tag ?client=gh-action-gauntlet-1.0.0, which only adds to a daily count of runs started from the action (at most one a day per runner address). Nothing about you is in it.
Off by default: the pull request comment (comment: true) and the claim link for a score page and badge (claim-link: true, a masked output). Runs stay anonymous unless an owner claims one.

A pass means your agent didn't fall for these traps on these jobs. It is a test result, not a statement that an agent is safe.

Run the Gauntlet by hand → The Trap Set →