We moved our agents' web reading into TechFabric Harness. Here's what we deleted, and what it caught

TechFabric engineers build AI agents that run in production for our clients and for ourselves, and the ones that write are only as honest as the pages they read. Two of ours read the web all day: the writer behind the techfabric.com blog, and the agent that keeps our Databricks Field Guide current. Every claim either of them publishes is checked against those pages first.

A week ago each agent read pages its own way. When Cloudflare shipped web_fetch on 11 October we wired it into both the next day, as a direct call from each agent. It worked, and it left us with the same reading code in two places, a workaround for running it outside a Worker, and a version pin to stop a dependency from breaking our model gateway.

So we did what we'd tell a client to do with shared plumbing. We moved it into our agent framework, TechFabric Harness, released it, and put both agents on it. This is what we deleted, what we gained, and the security gap the move closed.

What we deleted

In each agent, three things went away:

• The direct fetchWeb() call. Each agent called Cloudflare's reader itself, with its own options and its own error handling.
• A REST shim. fetchWeb() expects a Workers AI binding. Our agents run in GitHub Actions, not in a Worker, so each carried a hand-written object that impersonated the binding by calling Cloudflare's /ai/tomarkdown endpoint. The blog writer also carried a test for it.
• An .npmrc pin. Adding Cloudflare's agents package would have upgraded the model library under our framework, so both agents pinned it back. More on why below.

Each agent now asks the framework for a page instead:

const fetchPage = cloudflareWebPageFetcher({
  accountId: account,
  apiToken: token,
  userAgent: READER_UA,
  timeoutMs: 30_000,
});
const page = await fetchPage({ url });

An agent that wants the model to search and read for itself can register both tools in one line:

const tools = cloudflareWebTools(cloudflareWebToolsFromEnv(process.env));

Ours don't, deliberately. Both agents read pages in their own code and hand the text to the model, so the model never decides on its own which URL to open. That's a choice we'd make again for any agent that publishes.

The line counts don't shrink much, about +190 and −102 in each agent, and that's the honest part of the story. Most of what we added is security tests.

What it caught

Before the move we ran a simple probe. We gave each agent a public hostname that resolves to its own machine, 127.0.0.1.nip.io, pointing at a local page that said "INTERNAL SECRET".

The old direct call read it. The framework's reader refused it with POLICY_DENIED.

That's a server-side request forgery hole, the kind that matters the moment an agent reads a URL somebody else chose, like a link in an email or a source in a feed. TechFabric Harness now checks where a name actually resolves before it reads, applies the agent's network policy on every redirect, and holds the page request and its Markdown conversion to one 30-second deadline. It took fourteen review rounds to get the policy applied everywhere a request can leave, and every gap found along the way has a test that fails on the old code.

We're not calling it finished. A DNS rebinding trick, where a name resolves one way when it's checked and another way when it's read, can still reach a private address, on the framework's reader and on the plain fallback our agents keep for outages. It's open in the TechFabric Harness tracker. It isn't a regression, since neither agent checked DNS at all before, but we'd rather say so than leave it out.

The upgrade we'd been avoiding

Both agents had been pinned to TechFabric Harness 6.2.0 since a fresh install broke their connection to Cloudflare's AI Gateway earlier this month. Taking the web tools meant leaving that pin, so we went looking for the cause.

It wasn't where we expected. Version 6.5 pulled in a newer Anthropic SDK that, when a call carries no API key, goes looking for credentials in the developer's home directory. Our calls carry no key on purpose, because the gateway holds it. On a laptop the call failed before it sent anything, with "Credentials file not found". On a machine with a profile, it would have attached that developer's Anthropic token to a request bound for Cloudflare. CI has no profile, so CI never showed it.

The fix was to move forward, not back, because the next version of the model library never does that lookup. We released the framework with it, confirmed real gateway calls with the bad profile still in place, and moved both agents up.

The extra 78 packages come with the Cloudflare integration's own dependencies. That's a real cost, and a lockfile diff is the first thing we'd read before taking any SDK into an agent that publishes.

What it didn't change

Speed. We read five real sources twice, before and after, and the medians landed within network noise of each other. The text the agents receive is identical. We'd rather not claim a speed-up we can't measure.

Both agents passed their canaries after the merge. The Field Guide's took 2 minutes 26 seconds and the blog writer's 4 minutes 44 seconds, each reading real pages and producing a real decision.

Why it belongs in the framework

The work we do for clients is mostly agents that act on other people's data, and every one of them reads something it didn't write. A reader that refuses private addresses, honours a network policy on every hop and has one deadline isn't a feature of one agent. It's the floor every agent should stand on. TechFabric Harness is where we keep that floor, so the next agent we build starts there instead of rediscovering it.