AI guides

Why a Gemini agent shed specialist tools: trade-offs in Bash and media viewing

By 郎瀚威 Will3 viewsSource: 郎瀚威 Will
Bash and media-viewer agent harness comparisons

Philipp Schmid describes an agent using Bash and a media viewer. Across over 350 tasks, harness scores were close while tokens and costs differed. Each task ran once, so score differences still require repeated experiments.

On November 11, day two of AI Engineer Code Summit (AIE CODE 2026), Google DeepMind MTS Philipp Schmid discussed a personal habit rather than a new model: roughly every six months, he rebuilds his coding agent from scratch and asks how much of its scaffolding the model still needs.

This time, sixteen tools became Bash plus a media viewer. System instructions and tool definitions shrank from 12,400 tokens to 574.

The headline result: he tested four harnesses—the tools, rules and prompts around a model—on over 350 coding, terminal and office tasks. Scores ranged from 69.4% to 74.9%, but costs varied widely: Bash plus image viewing cost $2.33 per task with 89% cache hits; the others cost $3.01–5.33 with 72–84% hits. Three takeaways:

1.Scores obscure differences that bills reveal. With the same Gemini 4 Argon, tokens per task ranged from 3.82 to 5.30 million across harnesses, a 39% difference.

2.Stronger models benefit more from Bash. Across six small tasks, the default-tools/Bash token ratio was 1.4–3.0 for Gemini 4 Carbon, versus 0.95–1.7 for the six-month-old 3.5 Flash. Carbon chained 2.2–3.7 commands per Bash call on average, compared with Flash’s 1.3–1.8.

3.Developers should remove scaffolding before models remove it for them. In his words: give the model a computer, not a toolbox.

Who is he?

Philipp Schmid, @_philschmid on X with about 127,000 followers, is based in Nuremberg. His profile says “Agents & Gemini API, MTS @GoogleDeepMind.” Previously Hugging Face technical lead and an AWS Machine Learning Hero, he blogs at philschmid.de, mostly about experiments he has conducted himself.

This talk concerned his personal practice, not the company: rewrite a coding harness approximately every six months and ask which parts have become unnecessary burdens. This time, he presented the entire process of carrying out the experiment: how the tools were removed, what the Bash tool looked like afterward, what was used to evaluate it, and what the evaluation found.

1. One task: ten rounds versus two

He began by renaming charge_customer’s amount parameter to amount_cents and updating all callers, using two approaches:

Article illustration

The same rename: file tools take ten rounds; Bash takes two. Source: AIE CODE 2026 livestream.

•Approach A: file tools, ten rounds. Grep finds four matches; billing.py (52 lines), checkout.py (61) and retry_payment.py (47) are read separately; four edits modify one file at a time. A later round runs ruff check && pyright && pytest, followed by the final answer. Resending context each round totals roughly sixty thousand input tokens.

•Approach B: Bash, two rounds. The first writes inline Python with a dictionary mapping filenames to old strings, replacements and expected counts. It asserts text.count(old) == n before replacing and writing, exiting on mismatch, then runs ruff, pyright and pytest. The second gives the final answer. Input totals around seven thousand tokens.

The slide explicitly calls this illustrative and its token counts estimated. He describes roughly ten versus two rounds and about tenfold token use, accomplishing the same result.

Why would a model make that choice? He explained the process of post-training—training that continues after the base model: one task, one model, a set of tools, a runtime environment, and a verifier that assigns scores. The model tries to solve the task itself and calls various tools, then is updated according to the scores the verifier provides. Nobody tells them which tool to use; they receive scores. More elaborate tools may therefore induce detours.

2. Rebuilding: sixteen tools and many rules become one tool

Starting point: a 2025 coding agent

Article illustration

A 2025 coding agent’s sixteen tools and system rules. Source: AIE CODE 2026 livestream.

The tools shown are Task, Bash, BashOutput, KillBash, Glob, Grep, LS, Read, Edit, MultiEdit, Write, NotebookEdit, WebFetch, WebSearch, TodoWrite and ExitPlanMode. Instructions contain repeated requirements:

• Avoid find and grep; search with Grep, Glob or Task instead;

• Avoid cat, head, tail and ls; read with Read and LS;

• Use the tools very frequently to record progress;

• Keep answers concise, under four lines.

He says this worked. A timeline shows February 2025’s fixed query/search/reflect/answer workflow, ordered by code; October 2025’s dozen-plus tools and six MCP servers, with 47,000 tokens of tool definitions alone; December 2025’s roughly ten-tool coding agent with separate read/edit/search/command tools; and today’s one-tool version with under six hundred initial tokens. Models absorbed successive layers: fixed order, planning and todos, retrieval, then file and search tools.

End point: just Bash

Article illustration

The single-tool loop: a prompt enters, the model chooses commands for Bash, output returns to the model, and the loop continues until an answer. Source: AIE CODE 2026 livestream.

The complete tool definition appears on the slide:

•Name: bash.

•Description: execute a command in the current workspace. Each call starts a clean subshell in the working directory. After 120 seconds, an unfinished command continues in the background and returns a PID and log path. Use tail for logs and kill to stop. Return only the first and last four hundred lines, saving full output to a file.

•Parameters: command, the command to run; description, a sentence explaining the action in the UI; and async_after_ms, default 120000 milliseconds before backgrounding. The first two are required.

“Reading, searching, editing and testing are just commands.” The environment includes cat, sed, rg, python, git, pytest, curl and other agents.

He was very specific in the talk about handling long-running tasks: when a command exceeds the configured number of milliseconds, the tool call returns first and the model receives a local file handle. It can return later to check whether the process is still running. This is particularly useful for web development: logs are written to a file, and the model reads them itself.

Subagents, background jobs and memory: Bash plus files

Article illustration

Replacements for three removed dedicated tools. Source: AIE CODE 2026 livestream.

The alternatives are illustrated by three terminals:

•Long-running tasks: pnpm build exceeds 120 seconds, returning “running in background, pid 4182, log .ga/tasks/bash-6a74.log”; the model later reads it with tail -n 20.

•Subagents: the agent is itself a CLI, so Bash invokes it directly: ga run "Review src/auth for race conditions" > notes/review.md. It runs in the background, then cat notes/review.md retrieves results. An agent can call itself without a separate subagent tool.

•Plans and memory: cat >> notes/plan.md appends Markdown todos such as - [x] reproduce the bug and - [ ] fix tax rounding. cat notes/plan.md checks progress. The filesystem records work without dedicated planning or memory tools.

System instructions: 12,400 tokens become 574

Article illustration

Each dot represents about a hundred tokens of system prompt and tool definitions. Source: AIE CODE 2026 livestream.

He counted how many tokens several agents have to read before reading the user’s message—system prompts plus tool definitions—using public prompts and locally installed versions:

• Mid-2025 coding agent: sixteen tools, 12,400 tokens.

• Open-source coding agent, 2026: eight tools, 8,500 tokens.

• Vendor-native coding CLI, 2026, optimized for its own model: five tools, 3,000 tokens.

• Minimal coding agent, 2026: four tools, 1,000 tokens.

• His “single-tool” agent, Bash plus image viewing: two tools, 574 tokens.

How much information formerly written into prompts is now already internalized by models?

The one thing Bash cannot replace: images

Bash cannot view images. cat screenshot.png returns binary data, flooding context without showing the image. He added view_media to return images in model-visible form. The final harness has two tools: bash and view_media.

3. Measurement: one trajectory, then over 350 tasks

After removing tools, he tested whether the change was right at two levels.

Level one: call-by-call comparison on the same task

Article illustration

Each column is a model call; each block a tool call or chained shell command. Colors distinguish reads, searches, edits, tests/lint/types, todos and other shell actions. Source: AIE CODE 2026 livestream.

The DeepSWE 1.1 task adds aiomonitor snapshots, snapshot CLI and web interfaces and a /snapshots page for capturing and comparing task states over time. Gemini 4 Carbon ran at medium reasoning; both attempts passed.

•Eight structured tools, including read, edit, grep and todos: 47 model calls and 4.2 million tokens. The row of blue at the beginning represents parallel file reads, followed by a long sequence of orange edits, modifying one file at a time. Updates to the to-do list appear between them in red, with tests and lint in green at the end.

•Bash alone: eighteen calls, 1.7 million tokens.

He emphasizes that the model reads in parallel but edits one location per round, repeatedly, making changes slow and expensive.

Level two: over 350 tasks, four harnesses

Article illustration

Gemini 4 Argon on over 350 coding, terminal and office tasks; scores average three evaluation sets. Source: AIE CODE 2026 livestream.

His own set contains approximately 350 coding, knowledge-work and terminal tasks, comparing four harnesses with Gemini 4 Argon:

• Bash plus viewer: bash and view_media; 72.7%; 3.82 million tokens per task; 1.0 baseline ratio; 89% cache hits; $2.33 per task.

• Bash only: bash; 74.9%; 5.30 million tokens; 1.39 ratio; 72% cache hits; $5.33 per task.

• Structured A: eight file tools; 69.4%; 5.19 million tokens; 1.36 ratio; 76% cache hits; $4.68 per task.

• Structured B: seven file tools; 72.7%; 5.01 million tokens; 1.31 ratio; 84% cache hits; $3.01 per task.

Scores look similar with some variation. He acknowledges each task ran only once, so differences may be noise and repeated runs are needed.

Tokens and costs differ: the other setups cost 1.3–2.3 times the Bash-plus-viewer baseline, which he verbally calls “almost twice as cheap.” Its cache hit rate is highest at 89%. His final slide also highlights 34% fewer tokens when replacing Argon with Carbon in the same Bash-only harness, and 39% more tokens for the heaviest versus lightest harness with one model.

His warning: scores alone will not reveal it. Strong models work around complex scaffolding and still finish, but spend extra rounds and tokens.

4. Stronger models chain more commands per call

He showed an accelerated recording in Pi, whose toolset is easily customized, comparing 3.5 Flash and Carbon with default tools or Bash alone. The shared prompt renames charge_customer’s parameter, updates callers and tests to use order.total_cents and job.amount_cents rather than deprecated aliases, requires ruff, pyright and pytest to pass, requests a minimal diff and a one-sentence reply.

Article illustration

Each row is a task: default tools on the left, Bash only on the right. Source: AIE CODE 2026 livestream.

Six tasks were tested: parameter renaming, rebranding 42 documents, bisecting flaky tests, counting the most frequent 5xx log errors, aggregating 59 CSV files and writing an incident report.

•Carbon: default/Bash token ratios were 1.4–3.0. Renaming improved most: ten calls became six, and 65,300 tokens became 21,500.

•3.5 Flash: renaming remained 22 calls versus 22, with 114,000 versus 113,000 tokens. Rebranding used slightly more (a 0.95 ratio); only data tasks reached a 1.7 ratio.

• Average commands per Bash call: Carbon 2.2–3.7, Flash 1.3–1.8.

Command sequences: two paths through the same task

Article illustration

Both use Bash only: 3.5 Flash has 22 model calls and 112,700 tokens; Carbon seven calls and 28,900. Source: AIE CODE 2026 livestream.

Flash makes 21 Bash calls and 31 commands, averaging 1.5:

• It starts with git status, rg "charge_customer" and pytest && ruff check && pyright;

• Then cat reads billing.py, client.py, checkout.py and retry_payment.py separately;

• Four python3 -c calls edit one file each, calls ten through thirteen;

• At the end it repeats git diff, find . -name "__pycache__" -exec rm -rf {} + and pytest && ruff check && pyright. Cache clearing happens twice and the full checks three times.

Carbon makes six Bash calls and thirteen commands, averaging 2.2:

• Call one combines rg "charge_customer" with find . -maxdepth 4 to establish scope;

• Call two uses head -n 100 on pyproject.toml and all .py file beginnings;

• Call three defines inline Python function replace_in_file(路径, 旧, 新), asserting assert old in text and failing if absent. It calls the function for four files, then runs git diff && ruff check && pyright && pytest in the same call—five commands;

• Calls four through six perform git status --short, git status && ls -la and ruff check src tests && pyright src tests as final checks.

With default tools, Carbon already parallelizes and reduces calls. With Bash alone, it writes one Python script to make all changes at once. File reading also becomes smarter: instead of reading every file in full, the model first uses ripgrep to locate the places that need changing, then makes the changes in one go. Flash behaves differently. It uses Bash as “one command per round,” reading once and writing once, so whether dedicated read and write tools are available makes no difference to it.

His conclusion: stronger models combine more commands per call and gain more from Bash. That explains why the same approach saves little for a model from six months earlier.

5. What his blog adds

Outside the talk, on September 1 he posted “Another Bitter Lesson is coming” on X, linking to a blog titled “How Foundational Models Became Superhuman in Bash.” It contains three examples he selected from the trajectories:

•Multi-file renaming: Python checks every replacement count, exits without writing on mismatch, modifies four files, runs ruff, pyright and relevant tests, then prints a concise diff.

•Bisecting a flaky regression: a temporary Git worktree leaves the local repository untouched. Each commit runs five seeds and is considered good only with fewer than three failures, avoiding misleading one-off failures.

•Compressing production logs: Python streams rotated gzip logs into SQLite for joins, P95 latency ranking and error categorization. Only five JSON summaries enter model context.

He also made a candid admission in that post: just a few months earlier, he himself had advocated robust, atomic tools rather than letting models directly use cat, sed and echo. His reasons were that the shell could overwhelm the context, produce unclear errors, and damage files through quoting mistakes. He said those limitations still hold, but today’s models can combine small patches, Git commands and tests well enough that the key question has become harnesses must replace the protections atomic tools supplied, including output truncation, exit codes and timeouts, enforcing path and network restrictions outside the model, and starting and stopping long-running processes. This matches the Bash tool definition above: keeping the first and last 400 lines of output, saving the full output to a file, and moving commands to the background on timeout all provide these safeguards.

His closing line: a smaller interface and a larger action space.

6. The next bet: rebuild again in six months

Article illustration

A stronger model cuts tokens 34% in the same Bash-only harness, Argon to Carbon; a heavier harness adds 39% for the same model. Source: AIE CODE 2026 livestream.

He ended with directions for another rewrite rather than declaring the work finished.

•Models will keep absorbing the harness. He expects more comparisons in 2027 to focus on “how many tokens it takes to solve a task.” From Argon to Carbon, better models use fewer tokens per task. He considers that good for everyone, because it means people can use the models more.

•Include token efficiency in evaluation. Models are strong enough to make harness accuracy look similar, working around complexity through extra rounds.

•Code mode may come next. He sees movement toward TypeScript or Python rather than Bash. An MCP supporter, he notes structured outputs enable better scripts with more actions. Bash is hard to review; TypeScript offers an abstraction for examining commands. His final slide proposes bash for files/processes and exec(code) for everything else, typed tool inputs/outputs for composition, and host-held keys and policies outside the sandbox. A TypeScript example lists Linear projects, fetches issues in parallel, filters unassigned items and archives them in parallel. Only the final “Archived N issues” enters context. His November 1 blog is Why Agents Are Soon Only Going to Have 1 Tool; on November 2 he predicted MCP’s resurgence in 2027.

His final three messages:

1.Delete your code before the model does it for you.

2.Give the model a computer, not a toolbox.

3.With every new model, ask more boldly: what if the entire harness were removed and only one tool remained?

What readers can take away

There is no new algorithm or large dataset here: he removes his own code piece by piece and measures again. Sixteen tools become Bash plus a viewer, 12,400 prompt tokens become 574, scores differ by under six points across over 350 tasks and cost is $2.33 versus $3.01–5.33.

Agent developers can apply these checks immediately:

•Look beyond scores. Put each task’s token count, number of calls, cache-hit rate and dollar cost alongside its score. In his table, the setup with the highest cache-hit rate is also the cheapest.

•With every new model, ask whether it still needs each tool. He rewrites every six months, measures and then decides what stays.

•Restore protections as tools disappear. Truncation, backgrounding after timeout and assertions before commands must accompany Bash replacing dedicated tools.

•Interpret data honestly. Each task ran once and score differences may be noise. His conclusion is lower cost, not guaranteed higher accuracy.

• Livestream, from 2:18:55: https://www.youtube.com/watch?v=ZZ19zhYnDKY&t=8335s

• How Foundational Models Became Superhuman in Bash, September 1, 2026: https://www.philschmid.de/superhuman-bash

• Why Agents Are Soon Only Going to Have 1 Tool, November 1, 2026: https://www.philschmid.de/code-mode

• Posts: https://x.com/_philschmid/status/2094799789636407564 ; https://x.com/_philschmid/status/2117278154918092817

This article was originally published by Lang Hanwei Will. Author: Lang Hanwei Will.

Related topics

After reading

Join the AINRK community

Read how to join the community and connect with people working in AI.

Community information (opens in a new tab)