Cache MCP Tool Responses: Stop Re-Fetching the Same Page

Stop Your MCP Server From Re-Fetching the Same Page Every Turn

Every MCP server that wraps a scraping tool has the same default behavior: the agent calls get_page, the server fires an HTTP request, the response comes back. Next turn, the agent calls get_page again — often for the exact same URL — and the server fires another HTTP request. Nobody decided this should happen. It’s just what happens when your tool handler is a stateless pass-through.

Watch a long agent session with tracing enabled and the waste is obvious. The agent reads pricing docs in turn 3, references them in turn 5, quotes them in turn 7, and each of those turns triggers a fresh fetch of the identical page. You pay for the request, you wait for the latency, and the page hasn’t changed. A TTL cache in the tool handler fixes all three problems at once, and it’s maybe 60 lines of Python. This post builds one, then extends it with stale-while-revalidate, invalidation, and failure handling.

Why Agents Re-Fetch the Same URL Every Turn (and What It Costs)

Agents don’t re-fetch because they’re dumb. They re-fetch because your tool gave them no reason not to. The tool contract is “call me, get the page,” and the model has no memory of HTTP responses between turns — its context holds the extracted answer, not the raw page, so when it needs to re-check a detail, it calls the tool again.

Here’s a trace from a real session, captured the way I describe in Debug MCP Tool Calls: Trace What Your Agent Fetched:

[turn 1] tool_call: mcp__scraper__get_page
          args: {"url": "https://example.com/docs/pricing"}
          latency: 812ms

[turn 2] tool_call: mcp__scraper__get_page
          args: {"url": "https://example.com/docs/pricing"}
          latency: 794ms

[turn 3] tool_call: mcp__scraper__get_page
          args: {"url": "https://example.com/docs/pricing"}
          latency: 837ms

Three turns. One unique URL. Three full fetches. The page is static documentation; it did not change between turn 1 and turn 3.

Now scale that across a session. Say a 10-turn task touches the same documentation set repeatedly, with 8 of the 10 tool calls hitting URLs already fetched earlier in the session:

Scenario	Fetches	Per-call latency	Total tool time
No cache (cold fetch every call)	8	~800ms	~6.4s
TTL cache (2 unique cold, 6 hits)	2	~2ms on hit	~1.6s

The 800ms figure is a plain HTTP fetch through a scraping API with no JS rendering — add use_js_render for an SPA and you’re looking at 3-5 seconds per call, which means a chatty agent session can burn 30+ seconds purely on redundant fetches. Cached hits return in single-digit milliseconds because the response never leaves your process.

There’s a second cost that’s less visible: tokens. If your tool returns the full page each time, the agent’s context gets flooded with duplicate content, and you pay input-token prices on every copy. Caching doesn’t fix that by itself — pair it with Trim Scraped Pages Before They Flood the Agent Context — but it’s the prerequisite.

Designing the Cache Layer: TTL, Keying, and Scope Decisions

Before writing code, make three decisions: what the cache key includes, how long entries live, and where the cache physically lives.

Keying. The key must cover everything that changes the response: the URL and the fetch parameters. Two calls for the same URL with different formats or different proxy_country values are different responses. But the URL itself needs normalization first, because store.example.com/product/123?utm_source=newsletter and store.example.com/product/123 are the same page with different tracking garbage appended. If you key on the raw URL, your hit rate collapses.

from urllib.parse import urlsplit, parse_qsl, urlencode

TRACKING_PREFIXES = ("utm_", "fbclid", "gclid", "ref", "mc_cid", "mc_eid")

def normalize_url(url: str) -> str:
    parts = urlsplit(url)
    query = [
        (k, v) for k, v in parse_qsl(parts.query)
        if not any(k.lower().startswith(p) for p in TRACKING_PREFIXES)
    ]
    base = f"{parts.scheme}://{parts.netloc}{parts.path}"
    if query:
        return f"{base}?{urlencode(query)}"
    return base

For https://store.example.com/product/123?utm_source=agent&utm_medium=mcp, this returns https://store.example.com/product/123 — one canonical form, one cache entry, regardless of how the model mangled the URL this turn. Note the empty-query case: if stripping the tracking params leaves nothing behind, the trailing ? is dropped too, so you don’t end up with two “canonical” keys for the same page (/product/123 and /product/123?). And models do mangle URLs; they’ll paste the link with tracking params half the time and without them the other half.

TTL by content type. One global TTL is the lazy option and it’s wrong in both directions. Documentation pages change weekly — cache them for hours. Product pages on store.example.com change with price and stock — cache for minutes. I use a small table:

Content	TTL
Static docs, handbook pages	3600s
Product detail pages	300s
Search/listing pages	60s
Anything with a timestamp in the body	0 (no cache)

Scope. Where the cache lives matters more than people expect:

Option	Survives restart	Concurrency-safe	Dependencies
In-process dict	No	Yes, single-threaded; needs a lock otherwise	None
SQLite file	Yes	Yes (WAL mode)	stdlib
Redis	Yes	Yes	redis-py + a server

Here’s my opinion, and you may disagree: for an MCP server, Redis is usually overkill. Most MCP servers run as stdio processes — one per client session, one agent at a time. An in-process dict is correct, fast, and has zero operational surface. The catch is that stdio servers restart frequently, so every new session starts cold. If your agents run long multi-session workflows against the same URL set, promote to SQLite so the cache survives process restarts. Redis only earns its keep when you’re running the server over HTTP with multiple concurrent clients, which is a minority of MCP deployments today.

Implementing a TTL Cache Inside an MCP Tool Handler

The core is a wrapper: check the cache, return on hit, fetch and store on miss. Here’s a complete, working handler using the FastMCP style, with the actual page fetch going through a scraping API so the cache sits in front of the expensive part:

import time
import hashlib
import json
import threading
import requests
from mcp.server.fastmcp import FastMCP

mcp = FastMCP("scraper")

API_URL = "https://api.finedata.ai/api/v1/scrape"
API_KEY = "fd_your_api_key"

# key -> {"body": dict, "stored_at": float, "ttl": int, "etag": str | None}
# (failure-path entries add "negative": True — see the negative-cache section)
CACHE: dict[str, dict] = {}
CACHE_LOCK = threading.Lock()

DEFAULT_TTL = 3600

def cache_key(url: str, params: dict) -> str:
    blob = json.dumps({"url": normalize_url(url), **params}, sort_keys=True)
    return hashlib.sha256(blob.encode()).hexdigest()

def store(key: str, body: dict, ttl: int):
    """Write a positive cache entry stamped with the current time."""
    with CACHE_LOCK:
        CACHE[key] = {
            "body": body,
            "stored_at": time.time(),
            "ttl": ttl,
            "etag": None,
        }

def fetch_page(url: str, params: dict) -> dict:
    payload = {
        "url": url,
        "formats": ["markdown"],
        "only_main_content": True,
        "max_retries": 3,
        **params,
    }
    resp = requests.post(
        API_URL,
        json=payload,
        headers={"Authorization": f"Bearer {API_KEY}"},
        timeout=130,
    )
    resp.raise_for_status()
    return resp.json()

def cached_fetch(url: str, params: dict, ttl: int = DEFAULT_TTL) -> dict:
    key = cache_key(url, params)
    now = time.time()
    with CACHE_LOCK:
        entry = CACHE.get(key)
        if (
            entry
            and not entry.get("negative")  # failure entries are never hits
            and now - entry["stored_at"] < entry["ttl"]
        ):
            print(f"[cache] HIT  key={key[:12]} url={normalize_url(url)}")
            return entry["body"]

    print(f"[cache] MISS key={key[:12]} url={normalize_url(url)}")
    body = fetch_page(url, params)
    store(key, body, ttl)
    return body

@mcp.tool()
def get_page(url: str, ttl: int = DEFAULT_TTL) -> dict:
    """Fetch a web page as markdown. Repeated calls within the TTL
    are served from cache and are effectively free."""
    return cached_fetch(url, {}, ttl)

if __name__ == "__main__":
    mcp.run()

Two details worth calling out. First, the lock. Even a stdio server spawns threads for background work (you’ll add some in the next section), and dict mutation across threads is where intermittent corruption bugs come from. A threading.Lock costs nanoseconds; skip it and you’ll debug a heisenbug later.

Second, the TTL is a tool parameter, not a constant. That lets the agent itself decide freshness when it matters — the model can pass ttl=60 when it knows it’s looking at volatile data. Exposing cache controls to the agent feels weird at first. It works, because the tool docstring teaches the model the trade-off.

The log lines make the behavior verifiable at a glance:

[cache] MISS key=3f2a91c4b7de url=example.com/docs/pricing
[cache] HIT  key=3f2a91c4b7de url=example.com/docs/pricing
[cache] HIT  key=3f2a91c4b7de url=example.com/docs/pricing

Same key, one miss, two hits. That’s the whole win, compressed into three lines.

Serving Stale-While-Revalidate: Return the Cache Instantly, Refresh in the Background

A plain TTL has an annoying failure mode: the entry expires, the agent calls the tool, and now it blocks for a full fetch even though the cached copy is probably still fine. Documentation that’s 61 minutes old is not meaningfully different from documentation that’s 59 minutes old.

Stale-while-revalidate splits expiry into two thresholds. A soft TTL bounds how fresh the served content should be. A hard TTL bounds how stale you’ll tolerate. Between the two, serve the stale copy immediately and refresh in a background thread:

import threading

SOFT_TTL = 300    # serve from cache, refresh in background after this
HARD_TTL = 1800   # beyond this, block and fetch synchronously

def swr_fetch(url: str, params: dict) -> dict:
    key = cache_key(url, params)
    now = time.time()
    with CACHE_LOCK:
        entry = CACHE.get(key)

    if entry and not entry.get("negative"):  # never serve failure entries
        age = now - entry["stored_at"]
        if age < SOFT_TTL:
            print(f"[cache] HIT   {normalize_url(url)}")
            return entry["body"]
        if age < HARD_TTL:
            print(f"[cache] STALE {normalize_url(url)} — refreshing in background")
            threading.Thread(
                target=refresh, args=(key, url, params), daemon=True
            ).start()
            return entry["body"]

    body = fetch_page(url, params)
    store(key, body, HARD_TTL)
    return body

def refresh(key: str, url: str, params: dict):
    try:
        body = fetch_page(url, params)
        store(key, body, HARD_TTL)
        print(f"[cache] REFRESHED {normalize_url(url)}")
    except Exception as exc:
        print(f"[cache] refresh failed for {normalize_url(url)}: {exc}")

Both swr_fetch and refresh reuse the store(key, body, ttl) helper from the previous section, so the snippets concatenate into a single runnable module. Freshness is now decided by comparing the entry’s age (now - stored_at) directly against the SOFT_TTL and HARD_TTL constants — no derived expiry timestamps, no off-by-one-TTL arithmetic.

The flow for a store.example.com product page that crossed the soft threshold:

Agent            MCP server          Cache             Origin (store.example.com)
  |--get_page------>|                   |                    |
  |                 |--lookup---------->|                    |
  |                 |<-stale-but-edible-|                    |
  |<--response in ~2ms                 |                    |
  |                 |------- background refresh ------------>|
  |                 |<------------- fresh body --------------|
  |                 |--store, reset TTL-->|                  |
  |                 |                    |                    |
  |--get_page------>| (next call: clean hit)                 |

The agent never waits on the refresh. It gets a response in ~2ms, and the next call after the background thread finishes lands on a fresh entry. For price data where a few minutes of staleness is acceptable — which is most agent work; the model is summarizing, not arbitraging — this is the right default. If your task genuinely cannot tolerate stale data, drop the soft window to zero and you’re back to blocking behavior. That’s a per-tool decision, not a global one.

One honest caveat: background refreshes consume fetches even if nobody ever reads the refreshed copy. If an agent fetched a URL once and never returns to it, the SWR refresh was wasted spend. For expensive targets, weigh that against the latency win. In practice, URLs that hit the soft-expired state are usually URLs the agent keeps calling — that’s why they’re soft-expired — so the waste is small.

Invalidating Entries When the Source Page Changes

TTLs are a guess. Sometimes you can do better: ask the origin whether the page changed, for free.

Conditional requests are the classic mechanism. Store the ETag from a response, and on later validation, send it back with If-None-Match. If the server returns 304, the cached body is still valid and you paid for headers only — no body transfer, no re-parse. This works with a direct conditional GET against the origin:

import httpx

def validate_cached(url: str, etag: str) -> bool:
    """Returns True if the origin still considers our copy current."""
    r = httpx.get(
        url,
        headers={"If-None-Match": etag},
        timeout=10,
        follow_redirects=True,
    )
    if r.status_code == 304:
        print(f"[cache] 304 VALID   {url}")
        return True
    print(f"[cache] 200 CHANGED {url}")
    return False

A 304 here is the cheapest validation that exists — one round trip, zero body bytes. Not every site sends useful ETags, and some send ones that never change, so treat this as an optimization you enable per-domain after checking behavior, not a default.

The other invalidation path is explicit, and it’s the one I like more for agent workflows: give the agent an invalidation tool. Agents often know they just mutated the source. If the agent POSTed a form, submitted a review, or changed a setting on store.example.com, it can invalidate the affected page itself instead of trusting a TTL:

@mcp.tool()
def invalidate_cache(url: str) -> str:
    """Drop the cached copy of a URL. Call this after you have
    mutated data that affects the page, so the next get_page
    fetches fresh content."""
    key = cache_key(url, {})
    with CACHE_LOCK:
        removed = CACHE.pop(key, None)
    if removed:
        return f"invalidated: {normalize_url(url)}"
    return f"no cached entry for: {normalize_url(url)}"

The docstring does the heavy lifting. When the tool’s description says “call this after you mutate data,” well-behaved models actually do it. I’ve watched agents invalidate a product page immediately after a successful write action, unprompted, purely because the tool was available and its contract was clear.

TTL tuning remains the fallback for everything else. My rule of thumb: set TTL to roughly the interval at which a stale value starts producing wrong agent decisions, not the interval at which the page technically changes. A price that updates hourly but is only used for “roughly how much does this cost” answers can sit in cache for ten minutes without hurting anything.

Caching Failures and Non-200 Responses Without Poisoning the Cache

The naive cache stores whatever the fetch returned. Do that and one bad response gets served to every subsequent turn — the agent sees the same block page or empty body over and over, concludes the site is broken, and derails. A cache that stores failures is worse than no cache, because no-cache at least retries.

Guard what goes in:

MIN_BODY_BYTES = 500

def should_cache(status_code: int, content_length: int) -> bool:
    if status_code >= 500:
        return False
    if status_code in (401, 403, 429):
        return False   # transient blocks — retry later, don't freeze them
    if content_length < MIN_BODY_BYTES:
        return False   # empty or truncated body
    return True

The 500-byte floor catches a specific failure I’ve hit repeatedly: anti-bot interstitials and error pages that return 200 with a tiny body. If you cache those, every hit for the next hour serves a block page. If you want the full picture on detecting those, read Your 200 OK Responses Are Actually Block Pages — the short version is that status alone doesn’t prove you got real content, so check the payload size.

Separately, cache the fact of failure with a short expiry — a negative cache. When store.example.com is down, the agent will typically retry the same URL two or three turns in a row. Without a negative entry, each retry is a full 30-second timeout. With one, the second and third retries fail in milliseconds.

To make the fields concrete, here’s the shape of a successful fetch_page response — the negative-cache path below reads the same status_code and content fields that should_cache inspects:

# A successful fetch_page() response, abbreviated:
sample_response = {
    "status_code": 200,
    "content": "# Acme Widget\n\nThe Acme Widget costs $49.00 and ships in 2 days.",
}

NEG_TTL = 30  # seconds — long enough to stop a retry storm, short enough
              # to recover quickly

def fetch_with_negative_cache(url: str, params: dict) -> dict:
    key = cache_key(url, params)
    now = time.time()
    with CACHE_LOCK:
        entry = CACHE.get(key)
        if (
            entry
            and entry.get("negative")
            and now - entry["stored_at"] < entry["ttl"]
        ):
            remaining = entry["ttl"] - (now - entry["stored_at"])
            raise RuntimeError(
                f"recently failed, backing off for {remaining:.0f}s"
            )

    try:
        body = fetch_page(url, params)
    except Exception:
        with CACHE_LOCK:
            CACHE[key] = {
                "negative": True,
                "body": None,
                "stored_at": now,
                "ttl": NEG_TTL,
            }
        raise

    if should_cache(body.get("status_code", 200), len(body.get("content", ""))):
        store(key, body, DEFAULT_TTL)
    return body

Thirty seconds is deliberate. Long enough that an agent’s immediate retry loop hits the backoff. Short enough that a genuinely transient blip doesn’t poison a whole session. If the endpoint is still down after 30 seconds, the next attempt pays for one real fetch and re-arms the negative entry. This is a tiny circuit breaker — for serious degradation handling, see Trip a Circuit Breaker When Your Scrape Target Degrades, which extends the idea to per-domain state.

Measuring the Win: Latency, Token Usage, and Fetch Counts Before and After

Don’t ship the cache on faith. Instrument it, run a fixed benchmark task before and after, and compare. My benchmark is a 20-turn agent task that mixes reading docs, answering questions about them, and re-checking details — the shape of a real research session, not a synthetic loop.

Representative numbers from a local harness, plain HTTP fetches, no JS rendering:

Metric	Before cache	After cache + SWR
HTTP fetches issued	20	6
Mean tool latency	~810ms	~240ms
Cache hit rate	—	70%
Wall-clock session time	~19s	~7s

The mean latency doesn’t drop to 2ms because six calls still pay full fetch price; the mean is dragged down by fourteen near-instant hits. Wall-clock time is the number that matters — the agent’s turns stop stalling on I/O, and the whole session finishes almost three times faster. If your uncached fetches involve JS rendering at 3-4 seconds each, the gap gets much wider.

Instrumentation is trivial if you’ve been keeping counters alongside the cache:

STATS = {
    "hits": 0, "misses": 0, "stale_served": 0,
    "fetches": 0, "negative_hits": 0, "invalidations": 0,
}

def bump(stat: str):
    with CACHE_LOCK:
        STATS[stat] += 1

@mcp.tool()
def cache_stats() -> dict:
    """Report cache counters for this session."""
    total = STATS["hits"] + STATS["misses"]
    return {
        **STATS,
        "hit_rate": round(STATS["hits"] / total, 2) if total else 0.0,
        "entries": len(CACHE),
    }

Expose cache_stats as a tool and the agent can check it; also emit a summary line at session end so your server logs give you a per-session record:

[cache] session end: 14 hits, 6 misses, 70% hit rate, 20 tool calls, 6 fetches

Run the same 20-turn benchmark through the instrumented server before enabling the cache (force ttl=0 everywhere) and after. Two log lines, one comparison. If the hit rate is under 30%, your agent isn’t re-fetching much and the cache isn’t earning its complexity — check your key normalization before concluding anything else, because un-normalized URLs are the number one silent hit-rate killer.

Gotchas and Next Steps

A few things that will bite you if you’re not watching for them:

• stdio servers restart constantly. Every new client session can spawn a fresh process, which means an in-process dict starts empty every time. If your workflow is “one long session,” fine. If it’s “many short sessions over the same URL set,” move the cache to SQLite — same logic, persistent backing.
• Cache the fetch, not the extraction. If your tool uses extract_schema or extract_prompt for AI extraction, cache the raw fetched content and re-run extraction on hit only if the schema changed. Extraction output is non-deterministic and schema-dependent; caching it keyed on URL alone will serve the wrong shape when the agent changes what it’s asking for.
• Key includes fetch params. proxy_country, formats, and only_main_content all change the response. Fold them into the key hash or you’ll serve mobile-German markdown to a desktop-US request.
• Cap the cache. An unbounded dict grows for the life of the process. A simple eviction — drop the oldest entries when you cross, say, 500 — is enough; LRU is overkill here since agent access patterns are bursty and repetitive.
• Don’t cache authenticated pages across sessions. If a page was fetched with session-specific headers, its content may be user-scoped. Either exclude those URLs from the cache or key them with a session discriminator.

Next steps, in order of payoff: add the counters first so you have a baseline, then the TTL cache, then SWR, then the invalidation tool. Each layer is independently useful, and each one is verifiable from the logs. The whole stack is small — under 150 lines — which is exactly why it’s worth doing properly rather than bolting on a decorator from a library you’ll end up fighting when the stale-serving behavior isn’t quite right.

Your agents are going to keep re-fetching either way. The only question is whether the second through twentieth fetch of the same page cost you 800ms each, or 2ms.