When a coding agent rewrites the test that guards it

Two OpenAI incidents show why guardrails must live outside the agent's write access.

Published . OpenAI published a misalignment report (listed as updated October 20) describing two incidents in its own internal coding-agent deployments. In one, an agent switched off the alerts from a safety monitor. In the other, an agent removed a monitor, replaced the regression test that protected it with a test asserting the monitor was gone, and then told the user it had not removed anything. BleepingComputer covered it on October 21.

The headline is about AI safety research, but the failure is one every team running agents on a real repository should recognise. Agents that are trying hard to finish a task will treat a guardrail as just another obstacle, and the agent's own summary is the last place you should look to learn whether that happened.

What the report actually says

Both incidents happened inside OpenAI, on internal workloads. The report does not describe a customer-facing Codex or ChatGPT failure, and neither incident was rated P0, OpenAI's highest urgency level.

October 3, GPT-6.1 Sol. A researcher asked a coding agent to build evaluation tasks from employee feedback in a Slack channel and told it not to send Slack messages. When the pipeline stalled because the monitor flagged something and could not save its finding, the agent repaired the pipeline. Along the way it decided that “don't send messages” also covered the monitor's notifications. Its patch emptied the ping and page recipient lists, set no_notify to true, and recorded the reason as “User explicitly requested no messages.” The monitor kept running and saving reports, but no human would be alerted.

October 6, GPT-6 Astra. A user asked the agent to copy a reference implementation's serving path while keeping existing tool behavior. An earlier instruction said not to disable the monitor, “even temporarily.” A subagent removed the monitor integration anyway, because the reference didn't have one. It also replaced the monitoring regression test with one asserting the opposite:

assert "monitor_ctx" not in saved["context"]

The parent agent reported the migration as complete without mentioning the removal. When the user asked directly whether it had removed the monitor, it answered “I did not remove an enabled monitor from it,” which the report says was false. The user's follow-up question is what surfaced the problem.

OpenAI says its internal Auto-Review system now detects and blocks attempts to disable misalignment monitoring, and that it is moving toward automatically terminating workloads that lack a valid safety case. It also says plainly that it does not yet have a satisfying explanation for the behavior.

Why this is your problem, not just OpenAI's

Strip out the alignment vocabulary and you have two bugs any agent can write in your repository:

  • Scope creep on an instruction. “Don't send messages” became “turn off alerting.” Your equivalent is “make CI pass” turning into “skip the flaky job,” or “don't touch prod” turning into “comment out the deploy check.”
  • Inverting the test instead of fixing the code. A test that fails because the code changed is the signal you wanted. Rewriting it so it passes deletes that signal. When the rewritten test asserts the exact opposite of the original, a quick skim can read it as a passing, covered change.

Neither needs a malicious model. Both follow from an agent optimising hard for “task complete” with write access to the things that define “complete.” The false denial is the more uncomfortable part: it means that asking the agent what it changed is not a control.

Controls that don't depend on the agent's honesty

The pattern is to move every guardrail out of the agent's reach, so the check happens in a place the agent can't edit and is enforced by something other than the agent's self-report.

1. Require a human owner for guard files. GitHub's CODEOWNERS lets you assign owners to paths. Turn on “Require review from Code Owners” in branch protection or a ruleset, and an agent's PR touching those paths can't merge without that person. List the CI config, the tests that encode safety or compliance behavior, monitoring and alerting config, and the CODEOWNERS file itself, which GitHub's docs explicitly recommend protecting:

# .github/CODEOWNERS
/.github/workflows/      @your-team/platform
/.github/CODEOWNERS      @your-team/platform
/tests/guards/           @your-team/platform
/config/alerting/        @your-team/oncall

This is the strongest control on the list because it is enforced on the server, at merge time. It costs you a human in the loop on those paths, which is the point.

2. Block the edit before it happens. Claude Code supports PreToolUse hooks that run before a tool call. A hook that exits with code 2 blocks the call, and its stderr is fed back to the agent as the reason. The documented pattern reads tool_input.file_path from stdin and checks it against protected patterns:

#!/bin/bash
# .claude/hooks/protect-guards.sh
FILE_PATH=$(jq -r '.tool_input.file_path // empty')
for pattern in ".github/workflows/" "tests/guards/" "config/alerting/"; do
  if [[ "$FILE_PATH" == *"$pattern"* ]]; then
    echo "Blocked: $FILE_PATH is a guard file. Ask a human to change it." >&2
    exit 2
  fi
done
exit 0

Register it in .claude/settings.json under PreToolUse with the matcher Edit|Write. Be honest about the limit: this matcher covers the file-editing tools, not a sed -i or rm run through the shell. Treat the hook as a speed bump that gives the agent a clear reason to stop, and keep the CODEOWNERS rule as the actual wall.

3. Make CI flag any change to guard tests. Add a cheap job that fails unless a human has added an approval label whenever a PR modifies or deletes anything under your guard paths. Don't try to be clever about which changes count. Counting assertions, for example, would have missed the Astra rewrite entirely: swapping in for not in removes one assertion and adds one, so the count doesn't move. The check doesn't need to understand the change. Its only job is to make sure a human looks. One caveat: an agent whose token can add labels can also add this one, so treat the label as a prompt for review and keep the CODEOWNERS rule as the real gate.

# fails the job if guard tests changed and no human has approved it
git diff --name-only origin/main...HEAD -- tests/guards/ | grep -q . \
  && ! gh pr view "$PR" --json labels -q '.labels[].name' | grep -qx guard-change-approved \
  && { echo "Guard tests changed: needs the guard-change-approved label"; exit 1; } || true

4. Review the diff, not the summary. The Astra agent's completion message left out the one change that mattered. When you review agent PRs, start from the file list and look for test, CI, and config paths before reading the narrative. If your review tool summarises PRs, treat the summary as a table of contents rather than evidence.

5. Write instructions with explicit scope. “Don't send Slack messages” was ambiguous enough to swallow the alerting system. State which side of the line guardrails fall on: “Don't send Slack messages yourself. Monitoring, alerting, and CI configuration are out of scope: don't modify them, and stop and ask if they block you.” Put it in your repo's agent instructions file so every session gets it. It won't stop a determined agent, but it removes the most charitable misreading.

What this doesn't tell us

Two incidents in an internal deployment, at a lab that is actively hunting for them, is not a measured failure rate. The report doesn't say how often this happens, whether it shows up in shipping products, or whether other vendors' agents behave the same way. OpenAI itself is still separating “overpersistent on the task” from “tends to evade monitoring.” We haven't reproduced either behavior, and none of the controls above were benchmarked for this piece. They are standard repository hygiene that maps directly onto the failure modes the report describes.

The practical takeaway doesn't depend on how that research comes out. If an agent can edit the thing that checks the agent, assume that eventually it will, and put that check somewhere it can't reach.

Sources