Slow Scrape, No Timeout: MCP Progress and Cancellation

The Gap Between a Timed-Out Agent and a Still-Rendering Browser

Your MCP client hung up at thirty seconds. The headless browser kept rendering for another forty-five. That gap is the whole problem: the agent moved on, the user saw a timeout error, and nobody told the scraper to stop. You paid for the CPU, the proxy bandwidth, and the render — then threw the result away because there was no listener left to receive it.

This walkthrough fixes all three halves of that failure: progress notifications so the client can see long tool calls breathing, a rebudgeted timeout so the agent stops dying at an arbitrary wall, and real cancellation so the browser process actually exits. Everything is measured against a JS-heavy store.example.com product page, and there’s a local fixture at the end so you can rehearse the slow-path behavior without hammering a real site.

Why a 60-Second JS Render Outlives a 30-Second Tool Call Timeout

Most agent harnesses default to a 30-second tool-call timeout. A JS-rendered storefront page doesn’t care. Here’s the measured breakdown for one product page on store.example.com, rendered in a container with a throttled backend (4 vCPU, network shaped to 1.6 Mbps):

Clock	Agent side	Server side (measured)
t=0.0s	scrape_product dispatched	Navigate begins
t=2.1s	— waiting	TTFB + document received (2.1s)
t=8.9s	— waiting	Hydration bundle executed (+6.8s)
t=30.0s	Tool call timeout, request abandoned	Product-data XHR still in flight
t=52.6s	—	Data fetch completed (+41.6s, slow upstream API)
t=72.9s	—	Grid rendered, selector matched (+20.3s)
t=75.2s	—	Extract finished; result discarded, no listener

The tool code maps those raw events onto four stages: navigate, wait-for-selector (which absorbs hydration plus the data XHR), extract, and cleanup. The paired logs make the disconnect obvious:

[mcp-client] 12:04:31.204 ERROR callTool "scrape_product" timed out after 30000ms (reqId=42)

[mcp-server] 12:04:31.201 stage=navigate done in 2.1s
[mcp-server] 12:04:31.205 stage=wait-for-selector: hydrate done (6.8s), waiting on data XHR
[mcp-server] 12:05:23.811 stage=wait-for-selector done (52.6s elapsed)  <-- client timed out 22s ago
[mcp-server] 12:05:44.204 stage=extract done, 41 products
[mcp-server] 12:05:44.602 result discarded: request 42 has no pending handler; total=75.2s

The client error is one line and final. The server log shows a full minute of work after it. Multiply by every agent loop that retries on timeout — which most do — and one “failed” page becomes two or three browser sessions rendering in parallel for an audience of zero. If your tool wraps a hosted scraping API instead of local Playwright, the same abandonment happens remotely; we’ll cancel that too.

Emitting notifications/progress From a Slow Scraper Tool

The MCP spec’s answer to blind long calls is the progress token. The client attaches _meta.progressToken to the tool call; the server echoes that token in notifications/progress messages as work proceeds. Nothing about your scrape logic changes — you’re just adding a heartbeat.

Here’s a tool handler that reports at the navigate, wait-for-selector, and extract stages:

import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { z } from "zod";

server.registerTool("scrape_product", {
  title: "Scrape a JS-rendered product page",
  inputSchema: {
    url: z.string().url(),
    selector: z.string().default(".product-grid"),
  },
}, async ({ url, selector }, extra) => {
  // The client-supplied token; undefined if the client didn't ask for progress.
  const token = extra._meta?.progressToken as number | string | undefined;

  const beat = async (progress: number, message: string) => {
    if (token === undefined) return;
    await extra.sendNotification({
      method: "notifications/progress",
      params: { progressToken: token, progress, total: 1.0, message },
    });
  };

  await navigate(url);            // stage 1 of 4
  await beat(0.2, "navigate done (2.1s)");

  await waitForGrid(selector);    // stage 2 of 4 -- the long one
  await beat(0.5, "wait-for-selector done (52.6s)");

  const data = await extract();   // stage 3 of 4
  await beat(1.0, `extract done (${data.products.length} products)`);

  return {
    content: [{ type: "text", text: JSON.stringify(data) }],
  };
});

On the wire, a single tool call produces three notifications before the response:

{"jsonrpc":"2.0","method":"notifications/progress",
 "params":{"progressToken":17,"progress":0.2,"total":1.0,"message":"navigate done (2.1s)"}}

{"jsonrpc":"2.0","method":"notifications/progress",
 "params":{"progressToken":17,"progress":0.5,"total":1.0,"message":"wait-for-selector done (52.6s)"}}

{"jsonrpc":"2.0","method":"notifications/progress",
 "params":{"progressToken":17,"progress":1.0,"total":1.0,"message":"extract done (41 products)"}}

Two rules I hold to. First, emit per stage, not per percentage — a notification per scroll tick floods stdio and teaches the agent nothing. Second, if a stage drags, resend the same progress value with an updated message (“still waiting on data XHR, 30s in”) on a timer. A heartbeat that goes quiet for 50 seconds during the exact stage you most need visibility into is worse than no heartbeat at all, because it trains the client to trust silence.

Forwarding onProgress to the Agent Loop and Rebudgeting the Timeout

Progress notifications are useless if the client drops them. The TS SDK exposes onProgress in call options, and it also supports resetTimeoutOnProgress — each heartbeat resets the per-call deadline, with maxTotalTimeout as a hard ceiling. That combination is what makes it defensible to raise the timeout at all:

let lastBeat = Date.now();

const watchdog = setInterval(() => {
  const stallMs = Date.now() - lastBeat;
  if (stallMs > 10_000) {
    console.warn(`[watchdog] no progress for ${Math.round(stallMs / 1000)}s on scrape_product`);
  }
}, 2_000);

const result = await client.callTool(
  {
    name: "scrape_product",
    arguments: { url: "https://store.example.com/product/8841" },
    _meta: { progressToken: 17 },
  },
  undefined,
  {
    timeout: 60_000,                  // per-stage deadline, reset on each beat
    resetTimeoutOnProgress: true,
    maxTotalTimeout: 120_000,         // absolute cap for the whole call
    onProgress: (p) => {
      lastBeat = Date.now();
      console.log(`[progress ${p.progress}] ${p.message}`);
    },
  },
);
clearInterval(watchdog);

The config change once heartbeats exist:

 tool_call_timeout_ms: 30000
+tool_call_timeout_ms: 60000        # per stage, reset by each notification
+max_total_timeout_ms: 120000       # hard ceiling regardless of heartbeats
+progress_watchdog_ms: 10000        # warn, then cancel, on stalls
+reset_timeout_on_progress: true

An opinion here, because I’ve seen this go wrong: do not enable resetTimeoutOnProgress without maxTotalTimeout. An infinite-progress page — a carousel that never stops loading, an XHR poll loop — will heartbeat forever and your call never terminates. Some people prefer a single fixed deadline with no resets at all, which is simpler and fails faster; I think per-stage resets with a total cap is the better trade because it distinguishes “slow but advancing” from “hung.” But if a page legitimately needs more than 120 seconds, the right move is not a longer tool call. Make it an async job with a callback and have the agent poll or wait for a webhook — a blocking tool call that outlives the user’s patience has failed anyway, timeout or no timeout.

Honoring notifications/cancelled via the RequestHandler AbortSignal

Progress is half the contract. The other half: when the client gives up, it sends a cancellation notification, and the server must actually stop. The spec is explicit that this is cooperative — notifications/cancelled does not magically kill your handler. In the TS SDK, every handler receives extra.signal, an AbortSignal that fires when a cancellation arrives for that request.

The cancellation message itself, exactly as the client sends it:

{"jsonrpc":"2.0","method":"notifications/cancelled",
 "params":{"requestId":42,"reason":"agent tool deadline exceeded"}}

Note it carries requestId, not the progress token. A common bug is watching for the wrong identifier and wondering why nothing ever fires.

Server side, check the signal between stages and wire an abort listener that tears down anything in flight:

async ({ url, selector }, extra) => {
  const stage = (n: number, name: string) => {
    if (extra.signal.aborted) {
      throw new ScrapingCancelled(`stage ${n}/4 (${name})`);
    }
  };

  const controller = new AbortController();
  const onAbort = () => controller.abort();
  extra.signal.addEventListener("abort", onAbort, { once: true });

  try {
    await navigate(url);
    stage(2, "wait-for-selector");
    await waitForGrid(selector, controller.signal);
    stage(3, "extract");
    return { content: [{ type: "text", text: JSON.stringify(await extract()) }] };
  } catch (e) {
    if (e instanceof ScrapingCancelled) {
      return { content: [{ type: "text", text: `cancelled: ${e.message}` }] };
    }
    throw e;
  } finally {
    extra.signal.removeEventListener("abort", onAbort);
    await cleanupBrowserWork();   // always runs, cancelled or not
  }
}

The server log for an early exit at stage 2 of 4:

[mcp-server] 12:04:33.910 cancellation received for request 42 (reason="agent tool deadline exceeded")
[mcp-server] 12:04:33.914 aborted at stage 2/4 (wait-for-selector), elapsed 9.7s
[mcp-server] 12:04:33.931 browser context released

When the tool fronts a hosted API rather than a local browser, the identical rule applies — the abort must propagate out-of-process or you keep paying for the abandoned run. For FineData’s async endpoints the cancel path is a job deletion:

extra.signal.addEventListener("abort", () => {
  void fetch(`https://api.finedata.ai/api/v1/async/jobs/${jobId}`, {
    method: "DELETE",
    headers: { Authorization: `Bearer ${process.env.FINEDATA_API_KEY}` },
  }).then(() => console.log(`remote job ${jobId} cancelled`));
});

Without that DELETE, the job finishes server-side and bills exactly like the local chromium that nobody collected. The pattern is the same shape as what we cover in Async Scraping at Scale: anything you submit, you must be able to retract.

Propagating the Abort Into Playwright So the Browser Actually Stops

Checking signal.aborted between stages is necessary but not sufficient. If the code is sitting inside page.waitForSelector when the cancel lands, the next stage check is forty seconds away. Playwright’s wait functions accept an AbortSignal option, so bridge the MCP signal into the browser layer directly:

const controller = new AbortController();
const onAbort = () => controller.abort(new Error("client cancelled"));
extra.signal.addEventListener("abort", onAbort, { once: true });

let page, context;
try {
  context = await browser.newContext();
  page = await context.newPage();

  await page.goto(url, { waitUntil: "domcontentloaded", timeout: 60_000 });
  await beat(0.2, "navigate done");

  // store.example.com grid: pending wait aborts within milliseconds of cancel
  await page.waitForSelector(".product-grid .product", {
    timeout: 90_000,
    signal: controller.signal,
  });
  await beat(0.5, "grid ready");

  const products = await page.$$eval(".product-grid .product", (nodes) =>
    nodes.map((n) => ({
      title: n.querySelector("h2")?.textContent?.trim() ?? null,
      price: n.querySelector(".price")?.textContent?.trim() ?? null,
    })),
  );
  await beat(1.0, `extract done (${products.length})`);
  return { content: [{ type: "text", text: JSON.stringify(products) }] };
} finally {
  extra.signal.removeEventListener("abort", onAbort);
  await page?.close().catch(() => {});
  await context?.close().catch(() => {});
}

The finally block is not optional decoration. waitForSelector aborting throws, the throw unwinds, and without the finally the context leaks — a page object whose renderer process outlives the request. Here’s the container before and after the fix, same cancel at the same point:

# BEFORE: abort checked only between stages
CONTAINER   CPU %    MEM USAGE / LIMIT
scraper     93.4%    612MiB / 2GiB    <- t+30s: client timed out
scraper     91.8%    601MiB / 2GiB    <- t+45s: chromium still rendering
scraper     0.4%     143MiB / 2GiB    <- t+75s: context finally reaped

# AFTER: signal wired into waitForSelector
scraper     92.9%    598MiB / 2GiB    <- t+9s: cancellation arrives
scraper     1.1%     151MiB / 2GiB    <- t+9.2s: chromium exited (~200ms later)

Two hundred milliseconds versus forty-five seconds of zombie rendering, per abandoned call.

Before/After Cost of Abandoned Scrapes on store.example.com

To size the waste, run a mixed workload through the harness from the next section: 100 tool calls against the fixture, 30 cancelled at a random point mid-flight. Numbers below are from that local run on a 4 vCPU worker; cost assumes $0.048 per vCPU-hour, which you should substitute with your own rate.

Metric (per abandoned call)	Before cancellation	After cancellation
Wall time after client gave up	45.2s	0.2s
Browser CPU-seconds	38.1	7.9
Peak resident memory	612 MB	598 MB (released in ~0.2s, not 45s)
Proxy egress	8.4 MB	1.1 MB
Est. compute cost per 1,000 abandoned calls	~$0.51 (10.6 vCPU-h)	~$0.11 (2.2 vCPU-h)

Peak memory barely moves — the render gets just as far as it did before the cancel point — so don’t sell this as a memory fix. The savings are CPU-time, egress, and, more importantly, concurrency headroom: zombie contexts are what turn a burst of agent timeouts into container OOM kills.

The measurement procedure, so you can reproduce it on your own stack:

# 1. Fixture + server up (next section)
node fixture.js & node server.js &

# 2. Mixed run: 100 calls, ~30% cancelled at t=10s
/usr/bin/time -v node run-mixed.js --calls 100 --cancel-rate 0.30 --cancel-at 10s

# 3. Sample container cost while the run executes
while run-mixed-pid-alive; do
  docker stats --no-stream --format "{{.Name}} {{.CPUPerc}} {{.MemUsage}}" >> stats.log
  sleep 1
done

# 4. Count zombie renderers: chromium processes alive >1s after each cancel wave
node assert-no-survivors.js

If you want these numbers attributed per URL and folded into real spend tracking rather than a one-off benchmark, the ledger approach in Build a Scrape Cost Ledger extends cleanly — add an abandoned outcome as a first-class cost category.

Rehearsing Slow Pages Locally With a Throttled example.com Fixture

You should not validate cancellation behavior against a live storefront — the latency varies, the page changes, and your test is unfalsifiable. Build a deterministic slow page instead:

// fixture.js -- example.com-shaped shell with a controllable data-fetch delay
const express = require("express");
const app = express();

const DELAY_MS = Number(process.env.FETCH_DELAY_MS ?? 45_000);

const shell = `<!doctype html>
<html><head><title>Example Store</title></head><body>
<div id="root"><p>loading…</p></div>
<script>
  // hydration stub, then a deliberately slow data fetch + render
  setTimeout(function () {
    var items = [];
    for (var i = 0; i < 24; i++) {
      items.push('<li class="product"><h2>Item ' + i + '</h2>' +
                 '<span class="price">$' + (9 + i).toFixed(2) + '</span></li>');
    }
    document.getElementById("root").innerHTML =
      '<ul class="product-grid">' + items.join("") + '</ul>';
  }, ${DELAY_MS});
</script></body></html>`;

app.get("/product/:id", (_req, res) => res.type("html").send(shell));
app.listen(4599, () => console.log("fixture ready on http://127.0.0.1:4599"));

Point the tool’s target at it — in the agent config, scraper_base_url: http://127.0.0.1:4599 instead of https://store.example.com — and FETCH_DELAY_MS gives you any stage duration you want, deterministically. Now the rehearsal script:

// rehearse.js -- 20 concurrent calls, half cancelled mid-flight
const { execSync } = require("child_process");
const { Client } = require("@modelcontextprotocol/sdk/client/index.js");

(async () => {
  const client = await connect();                       // stdio to your server
  const calls = [];
  for (let i = 0; i < 20; i++) {
    const p = client.callTool(
      { name: "scrape_product",
        arguments: { url: `http://127.0.0.1:4599/product/${i}` },
        _meta: { progressToken: i } },
      undefined,
      { timeout: 60_000, resetTimeoutOnProgress: true, maxTotalTimeout: 120_000 },
    ).catch((e) => e.message);

    if (i % 2 === 0) {
      setTimeout(() => client.cancelRequest(String(i), "rehearsal cancel"), 10_000);
    }
    calls.push(p);
  }
  await Promise.allSettled(calls);

  const survivors = execSync(
    "ps -eo etimes,comm | awk '$2 ~ /chrom/ && $1 > 1' | wc -l",
  ).toString().trim();
  console.log(survivors === "0" ? "PASS" : `FAIL: ${survivors} chromium survivors`);
})();

Sample output from a passing run:

launched 20 calls (10 scheduled to cancel at t=10s)
t=10.0s  cancellation wave sent (requests 0,2,4,…,18)
t=10.4s  chromium processes older than 1s: 0
t=61.9s  remaining 10 calls completed (24 products each)
PASS: no chromium process survives >1s after cancellation

The etimes > 1 assertion is deliberately strict. If a renderer lingers past one second, your abort path has a hole, and the production symptom will be an occasional container restart under load that nobody can explain.

Production Checklist: Heartbeat Cadence, Timeout Budgets, and Idempotent Cleanup

Everything above, distilled into config and a failure-mode contract. This is the block I ship with:

mcp:
  client:
    tool_call_timeout_ms: 60000        # per stage, reset on each notification
    max_total_timeout_ms: 120000       # absolute ceiling, heartbeats cannot extend
    progress_watchdog_ms: 10000        # stall warning; escalate to cancel at 2x
  server:
    heartbeat_interval_ms: 5000        # resend current progress on long stages
    stage_deadlines_ms:
      navigate: 20000
      wait_for_selector: 90000
      extract: 15000
    max_scrape_duration_ms: 180000     # server-side kill switch, client be damned
    context_idle_reap_ms: 30000        # reaper for anything the finally missed

And the contract each failure mode demands:

Failure mode	Required server behavior
Missed heartbeat (>watchdog with no progress)	Client cancels; server treats abort as terminal, releases context in finally
Duplicate cancellation	Abort listeners must be idempotent — { once: true } plus removeEventListener in finally
Cancel races with completion	Check signal.aborted immediately before returning results; cleanup runs either way
Orphaned browser context	finally is primary, periodic reaper is the backstop for anything that slips through
Progress emitted after cancel	Suppress notifications once aborted; clients ignore them, but don’t spray dead traffic
Cancel arrives before first stage	Return immediately with a cancelled marker; do not start the browser at all

The max_scrape_duration_ms server-side kill switch deserves emphasis. Client cancellation is advisory and networks drop messages; a server that trusts the client to always say “stop” will eventually host a two-hour render for a request that died at thirty seconds. Trust the signal, but keep the backstop.

Wrap-Up

The sequence is short: emit notifications/progress per stage, forward onProgress into the agent loop with a stall watchdog, raise timeouts only once heartbeats exist, and honor notifications/cancelled by bridging the handler’s AbortSignal into Playwright’s waits and any remote jobs you submitted. Measured on the fixture, an abandoned call drops from 45.2 seconds of orphaned rendering to a clean exit in about 200 milliseconds.

The gotchas that bite people: echo the progressToken exactly, remember notifications/cancelled carries requestId instead, always pair resetTimeoutOnProgress with maxTotalTimeout, and never let a stage run silent — a heartbeat that pauses during the slowest wait is the one failure mode this whole setup exists to prevent. Next step if your pages routinely exceed two minutes: stop stretching tool calls and move those targets to async jobs with webhooks, keeping the blocking path for everything that finishes fast.