A hosted coding agent can change what it will do without any change on your side. Last week Anthropic did it to Claude Code with two sentences, and a user spotted it in a rule sent from Anthropic's servers. If your team relies on an agent for real work, you need your own way to notice such a change on the day it lands, before a developer hits it mid-task. This article explains where those changes come from and walks through a small canary suite that catches them.

A rule arrived from the server on October 27

Game Developer reported on October 30 that an X user, Chasm, had spotted an update to Claude Code's rules "that was pushed sometime on October 27." The rule sits under a flag named "tengu_heron_brook" and reads: "Do build original games, games in a genre, and games from the user's own designs and assets. Do not build a recreation of a specific existing game, even if it is renamed or called original." Chasm reported that there is no way to disable it.

Anthropic confirmed the change. Game Developer quotes Addy Osmani, an Anthropic member of technical staff: "We added two sentences to Claude Code's prompt on Opus 5.5 to steer it away from recreating someone else's specific game because using Claude to infringe IP is against our usage policy". He added: "Original games, genre games, and games built from your own designs and assets are meant to keep working", and "if Claude turns down something that's yours, send it my way." Kotaku, reporting the same day, also names the flag and quotes the rule.

We take no side here on the policy itself. What matters for engineering teams is the mechanism: the change came from the vendor's servers, on a date the vendor chose, into a tool that teams use daily.

Four layers decide what a hosted agent does

When a developer asks a hosted coding agent for something, the answer depends on four things:

  1. The model, and which version of it the tool calls.
  2. The vendor's system prompt, the instructions the tool sends with every request.
  3. Server-delivered settings, flags like the one above that switch behavior on and off.
  4. Your own prompt and project files.

Only the last is in your repository. The first three can change between Monday and Tuesday with no deploy on your side, and without a note in your codebase. Often that's an improvement nobody notices. Sometimes it's a refusal, a different code style, or a new limit, and the first person to find out is whoever is mid-task when it happens.

Two words from the statement are worth noticing: "meant to". The vendor intends original work to keep working, and says so honestly. Whether your own tasks keep working is something only your own checks can tell you.

A canary suite: real tasks, run every morning

The fix borrows from mines and deploys: send something small ahead, and watch what happens to it. Pick five to ten tasks your team actually gives the agent, trim each to a minute's work, and run them on a schedule against the same tool your developers use. Record whether each one passed, and report only changes. A sketch of how we would write the runner, in TypeScript:

type Canary = {
  name: string;
  prompt: string; // a real task from your team, trimmed
  expect: (output: string) => boolean;
};

const REFUSED = /\b(can't|cannot|won't) (help|build|do|create)\b/i;

const canaries: Canary[] = [
  {
    name: "tile-editor-undo",
    prompt: "Add undo to the tile editor in src/editor/tiles.ts",
    expect: (out) => !REFUSED.test(out) && /undo/i.test(out),
  },
];

// Returns only what changed since the last run.
export async function runCanaries(
  ask: (prompt: string) => Promise<string>,
  last: Map<string, boolean>,
): Promise<string[]> {
  const changes: string[] = [];
  for (const canary of canaries) {
    const passed = canary.expect(await ask(canary.prompt));
    if (last.has(canary.name) && last.get(canary.name) !== passed) {
      changes.push(`${canary.name} now ${passed ? "passes" : "fails"}`);
    }
    last.set(canary.name, passed);
  }
  return changes;
}

Notice that the runner reports changes, not failures. A canary that has always failed tells you nothing new; one that passed yesterday and fails today is the signal. Save the tool's version and the model it reports alongside each run, so a change can be lined up with a release, or with no release at all, which is the case worth escalating.

Choose canaries near your edges: the game studio's own level editor, the security team's test harness, the legal team's contract parser. Those are the tasks a new rule is most likely to catch by mistake.

What a canary costs, and what it can't tell you

Each run spends tokens and a few minutes, every day, on work nobody uses. Keep the suite small and the tasks short.

Model output varies between runs, so a single failure can be noise. Run a failing canary three times before raising it, and write expect checks loose enough to accept any reasonable answer: a refusal and a missing keyword are good checks, an exact diff is not.

A canary tells you that behavior changed, not why. It won't show you the vendor's prompt, and it won't catch a change on a task you didn't think to include. If the cause is a vendor rule, your options are the ones the vendor offers (Osmani's invitation to send cases his way is one) or a second tool for that task. Plan which tool that would be before you need it.

If your team uses an agent for occasional one-off scripts, skip all of this. The suite pays for itself when an agent is part of a daily workflow that people depend on.

What to take from this

  1. A hosted agent's behavior has three layers you don't control: the model, the vendor's prompt and its server-side settings.
  2. Run five to ten of your own real tasks against it every day, and alert on changes, not on failures.
  3. Record the tool version and model with every run, so you can tell a release from a silent change.
  4. Know your second tool for each critical task before a rule makes you look for one.

If your team depends on a coding agent and wants a canary suite built around its real work, book a conversation.

Sources