What an 11-hour unattended agent run actually needed: lessons from Google's Argon experiment

The model did the exploring. The prompt, the repo guidelines, and the maintainers did the rest.

Published . Google's ADK team pointed Gemini 4 Argon at the Agent Development Kit's Python repository, about 205,000 lines across nearly 800 source files with almost 17,000 unit tests, and let it run overnight. The team's write-up on the Google Developers Blog is dated October 13. InfoQ picked it up on October 24, and it's getting passed around as proof that agents can now work through the night on real codebases.

The more useful read is the failure details. The first run went wrong in ways anyone who has left an agent alone with a mature repository will recognize. The second run worked because the team changed the setup, not the model. Of six optimizations the agent proposed, maintainers shipped two. Here's what happened and what to copy.

What Google ran

The goal was performance: make ADK add no noticeable overhead to an agent's run. The team used Google Antigravity's /goal mode, which the post says “preserves the goal across context compactions.” The goal prompt stays pinned while finished work collapses into summaries, so a long run doesn't forget what it was asked to do. The final run started at 9:11 PM and lasted about 11 hours. Before editing anything, Argon benchmarked four workloads 30 times each: package startup, a 200-step conversation, an app with 20 sub-agents, and a 12-node workflow. It then proposed six optimizations across 32 files, caught a bug in its own multi-agent cache, and ran all 16,892 open-source unit tests before stopping.

The shipped change is public as commit 1754b3e, titled “perf: cut time to first model request by 40% in long sessions.” It touches 22 files (+956/−101) and its message says plainly, “Argon generated these changes.”

Round one: what went wrong without guidance

The first attempt ran for about two hours with no repository instructions. It wasn't useless. Argon wrote a benchmark script seven minutes in, spent about 70 minutes measuring and reading before touching source, and found the real bottlenecks: session copying and package imports. Some of that code made it into the final commit. It also made four mistakes that are worth memorizing, because they're the default behavior of any agent on an unfamiliar repo:

  • It changed a public contract by accident. A custom hook “subtly altered its public JSON schema” for ADK's authentication model.
  • It missed edge cases. Its custom copier didn't handle mock event cases.
  • It tested locally, not globally. It ran 216 nearby unit tests out of almost 17,000.
  • It didn't show its work. The PR description had no benchmark numbers.

None of these are model-intelligence problems. They're missing-instructions problems, and the team fixed them with instructions.

Round two: what the team changed

Repository guidelines as skills. The team wrote its conventions into the repo's .agents/skills directory. Each skill is a short SKILL.md index pointing to focused references/ files, such as visibility.md for the private-module rule and imports.md for lazy-loading checks. The agent loads only what the current task needs. Argon read 12 guideline files before editing code. The design matters for long runs: a single giant instructions file eats context on every turn, while an index plus references costs almost nothing until it's needed.

A goal written as constraints and deliverables. The final prompt asked ADK to add “no noticeable overhead to any agent,” with hard invariants: “Keep what the model receives and the public API unchanged, and back each change with a test that would catch a regression.” It asked for one pull request “that the ADK maintainers would approve,” with before-and-after numbers in the description, and ended with “Don't merge it.” Every round-one failure maps to a line in that prompt.

A benchmark the agent didn't write. Results were measured with the team's holdout script, bench_turn.py, using medians of 6 to 12 runs, and the model received byte-identical requests before and after. An agent grading itself on its own benchmark is the performance equivalent of an agent editing the test that guards it, a failure mode we covered in our guardrail tampering piece.

The results, with their fine print

According to Google:

  • Pre-model overhead in long sessions dropped about 40%: the first model request after 50 turns went from 64 ms to 36 ms, and after 150 turns from 200 ms to 122 ms. The commit's benchmark table marks both of those measurements as run with garbage collection disabled, so expect different numbers under normal runtime settings. The gain also applies to InMemorySessionService. Database-backed session stores such as SQLite, Firestore, and Redis deserialize from storage, so their reads don't change.
  • Cold import google.adk got 21% faster, from lazy-loading live audio and video, OpenTelemetry, FastAPI, and Starlette.
  • A 158-turn run went from 65.6 s to 56.1 s.
  • Time from a new process to the first model request went from 3.07 s to 3.09 s, essentially flat, because deferred modules now load on the first turn. Google recommends a warm-up request before a production server takes traffic.

The benchmarks use an instant model and instant tools, so they isolate framework overhead. In a real agent loop, model latency dwarfs most of this. The savings matter for ADK because its runs can last hundreds of turns, not because any single call feels faster.

Four of six changes were rejected, and why

This is the section to show anyone who thinks overnight runs replace engineers. Google's summary is “the agent explores, the maintainer subtracts.”

  • Multi-agent event cache: 17% faster on a synthetic benchmark, but a weak memory reference could return stale data across sessions. That's a correctness bug that would only show up in production.
  • Telemetry check: read a private attribute on OpenTelemetry's tracer, which could break silently on a minor upgrade.
  • History formatting rewrite: harder to read, with no end-to-end gain on the holdout benchmark.
  • Workflow replay index: added state complexity that only helped synthetic replay tests.

One of the two accepted changes also needed a fix. Removing three top-level imports from llm_agent.py passed all 16,892 open-source tests but broke 25 internal Google test targets that patched those attributes. The shipped version keeps every public attribute patchable. A full test run is necessary, but your users' code is a test suite too, and the agent can't see it.

A playbook for your own overnight run

Nothing here is Antigravity-specific. The same structure works with Claude Code, Codex, or Copilot's coding agent.

  • Do a short scouting run first. Give the agent two hours and no special instructions, then read the transcript. Every mistake it makes becomes a guideline or a prompt constraint.
  • Write conventions as small, loadable files. Use an index that points to focused references, not a 3,000-line instructions file. Claude Code supports this shape through skills, and most agents read an AGENTS.md-style file you can keep short.
  • State invariants, not just a goal. Name what must not change (public API, wire format, model input), and require a regression test for every change.
  • Measure with something the agent can't edit. Keep the benchmark or eval outside the agent's write access, and require a baseline before the first edit.
  • Require the full suite. Spell out the full test command in the prompt, because “run the tests” gets read as “run the nearby tests.”
  • Make the output reviewable, and forbid merging. One PR, with numbers in the description, unmerged. Plan real review time the next morning: a 6-to-2 ratio means most of the work is deciding what to throw away.

Here's a goal skeleton built from those rules. It's our template, not Google's prompt:

Goal: reduce p50 latency of <workload> without behavior changes.
Invariants: public API, serialized formats, and requests sent
to external services stay byte-identical.
Before editing: run bench/holdout.py 10x and record the baseline.
Every change: add a test that fails if the change regresses.
Before stopping: run the full suite with `make test`, not a subset.
Deliver: one PR against main with before/after medians.
Do not merge. Do not edit anything under bench/.

What this doesn't tell us

This is Google reporting on its own model, using its own framework and its own holdout benchmark. Nobody has published an independent reproduction. The post gives no token or cost figures, so we can't say what an 11-hour run costs. It's also one run on one well-tested codebase with nearly 17,000 tests. A repo with thin coverage gives the agent far less to check itself against. We haven't run Argon or Antigravity for this piece. The analysis is based on Google's post, the public commit, and InfoQ's coverage.

The takeaway holds regardless of which model you use: an unattended run is only as good as the guardrails you write before you go to bed and the review you do when you wake up.

Sources