Skip to content
Technical 13 min read

Authorize an AI Agent to a Remote MCP Server

Wire OAuth 2.1 into an MCP agent client: discover the server's auth metadata, register dynamically, run PKCE, and refresh tokens when tool calls 401.

FE
FineData Engineering · Editorial Policy
|
On this page

Authorize an AI Agent to a Remote MCP Server

Your agent just tried its first tool call against a remote MCP server and got back a 401. Locally, over stdio, the same server worked fine — because a local server inherits your operating system trust and never asks who is calling. Over HTTP, that free ride ends. Every request needs a Bearer token, and something has to obtain, attach, refresh, and re-attach that token on the agent’s behalf.

That something is your client code. MCP delegates remote-server auth to OAuth 2.1: metadata discovery, dynamic client registration, authorization code + PKCE, and refresh handling. This tutorial wires the full loop into a Python agent client, then proves it works with a timed end-to-end run.

Where OAuth 2.1 Fits: Local stdio MCP vs. a Remote HTTP MCP Server

The difference between the two transports is trust, not features. A stdio server is a child process running as the same OS user as the agent — it reads the same environment and files, so there is no boundary to authenticate across. A remote server at store.example.com sits on a public network, trusts nothing by default, and demands a scoped, audience-bound Bearer token on every request.

Local stdio MCPRemote HTTP MCP (store.example.com)
Transportstdin/stdout, child processPOST /mcp over HTTPS
Trust boundarynone — same OS user as the agentpublic network; server trusts nothing by default
Credential scopeanything the launching user can readone Bearer token per request, scoped and audience-bound
Auth storyinherited OS permissionsOAuth 2.1: discovery, PKCE, refresh

Four actors participate. The agent is the OAuth client. The MCP server is the resource. The authorization server at example.com issues tokens. A human — the resource owner — approves consent in a browser. Even a headless agent needs the human for that one step; after it, the loop runs on its own.

Agent (client)      MCP server            Authorization server     Human
                    store.example.com     example.com              (browser)
  |-- GET /.well-known/oauth-protected-resource -->|
  |<-- { resource, authorization_servers } ---------|
  |-- GET /.well-known/oauth-authorization-server --------->|
  |<-- { issuer, endpoints, scopes } ------------------------|
  |-- POST /register ---------------------------------------->|
  |<-- client_id --------------------------------------------|
  |-- GET /authorize?...&code_challenge=... ---------------->|--> consent
  |                                                          |--> approve
  |<== redirect to http://localhost:8080/callback?code=... ===|<== browser
  |-- POST /token { code, code_verifier } ------------------>|
  |<-- access_token, refresh_token --------------------------|
  |-- POST /mcp  Authorization: Bearer ... -->|
  |<-- JSON-RPC tool result ------------------|

Discovering the Authorization Server via /.well-known/oauth-protected-resource

Do not hardcode endpoint URLs. The MCP spec points clients at the protected-resource document, which names the authorization server, which in turn publishes its own endpoints. Two GETs and you know everything.

GET https://store.example.com/.well-known/oauth-protected-resource
{
  "resource": "https://store.example.com",
  "authorization_servers": ["https://example.com"],
  "scopes_supported": ["mcp:read", "products:search"]
}

The authorization_servers entry is your hop two. Fetch the AS metadata document from that origin:

GET https://example.com/.well-known/oauth-authorization-server
{
  "issuer": "https://example.com",
  "authorization_endpoint": "https://example.com/authorize",
  "token_endpoint": "https://example.com/token",
  "registration_endpoint": "https://example.com/register",
  "scopes_supported": ["mcp:read", "products:search"],
  "response_types_supported": ["code"],
  "grant_types_supported": ["authorization_code", "refresh_token"],
  "code_challenge_methods_supported": ["S256"],
  "token_endpoint_auth_methods_supported": ["none"]
}

One validation matters more than the rest: issuer must match the URL you actually fetched, after normalizing trailing slashes. A mismatch means DNS tricks or a path-prefixed deployment, and tokens minted there will not be audience-bound to your MCP server. Refuse and stop.

import requests

def discover(store_url: str) -> dict:
    """Two-step discovery: protected-resource doc, then AS metadata."""
    pr = requests.get(
        f"{store_url}/.well-known/oauth-protected-resource", timeout=10
    )
    pr.raise_for_status()
    protected = pr.json()

    as_url = protected["authorization_servers"][0].rstrip("/")
    meta = requests.get(
        f"{as_url}/.well-known/oauth-authorization-server", timeout=10
    )
    meta.raise_for_status()
    auth_server = meta.json()

    if auth_server["issuer"].rstrip("/") != as_url:
        raise ValueError(
            f"issuer mismatch: {auth_server['issuer']} != {as_url}"
        )

    return {
        "resource": protected["resource"],
        "issuer": as_url,
        "authorize": auth_server["authorization_endpoint"],
        "token": auth_server["token_endpoint"],
        "register": auth_server["registration_endpoint"],
        "scopes": auth_server["scopes_supported"],
    }

I re-run this every session rather than caching metadata to disk. It costs about 120 ms, and stale endpoint URLs are exactly the bug you want to catch at startup instead of mid-session. If you point your client at the wrong well-known path you get an HTML 404 page back and a JSONDecodeError downstream — that failure mode has its own post.

Registering the Agent Dynamically at the Authorization Server’s /register Endpoint

Dynamic client registration (RFC 7591) removes the dev-console step: the agent POSTs its own metadata to the AS and receives a client_id on the spot. No pre-provisioned credentials, no ticket to file.

The agent is a public client — it cannot hold a secret, because anything compiled into or read by the agent process is effectively public. So it registers with token_endpoint_auth_method: "none" and lets PKCE compensate. The redirect URI is loopback, per RFC 8252.

{
  "client_name": "store-mcp-agent",
  "redirect_uris": ["http://localhost:8080/callback"],
  "grant_types": ["authorization_code", "refresh_token"],
  "response_types": ["code"],
  "token_endpoint_auth_method": "none",
  "scope": "mcp:read products:search"
}
{
  "client_id": "dc7f3a91-4b2e-4f8a-9c1d-2e6b8a0f5d31",
  "redirect_uris": ["http://localhost:8080/callback"],
  "grant_types": ["authorization_code", "refresh_token"],
  "response_types": ["code"],
  "token_endpoint_auth_method": "none"
}

Persist the registration next to the discovered endpoints, because you will need client_id on every refresh:

import json, pathlib, requests

def register(as_meta: dict) -> dict:
    resp = requests.post(
        as_meta["register"],
        json={
            "client_name": "store-mcp-agent",
            "redirect_uris": ["http://localhost:8080/callback"],
            "grant_types": ["authorization_code", "refresh_token"],
            "response_types": ["code"],
            "token_endpoint_auth_method": "none",
        },
        timeout=10,
    )
    resp.raise_for_status()
    reg = resp.json()
    state = pathlib.Path("~/.store-mcp-agent/registration.json").expanduser()
    state.parent.mkdir(parents=True, exist_ok=True)
    state.write_text(json.dumps(reg, indent=2))
    return reg

Two caveats. If registration_endpoint is absent from the AS metadata, that server does not support dynamic registration — fall back to a manually issued client_id and skip this step. And register the exact loopback URI you will serve; the AS matches redirect URIs byte-for-byte.

Running the Authorization Code + PKCE Flow from a Headless Agent

PKCE binds the authorization request to the token exchange, so a intercepted authorization code is useless without the verifier. Generate both with secrets:

import base64, hashlib, secrets

code_verifier = base64.urlsafe_b64encode(
    secrets.token_bytes(32)
).rstrip(b"=").decode()
assert len(code_verifier) == 43

code_challenge = base64.urlsafe_b64encode(
    hashlib.sha256(code_verifier.encode()).digest()
).rstrip(b"=").decode()

Fully rendered, the consent URL looks like this. Note the resource parameter (RFC 8707) — it tells the AS to mint an audience-bound token for store.example.com, which the MCP server will check. Omit it and you get a token that is technically valid but rejected by the server:

https://example.com/authorize
  ?response_type=code
  &client_id=dc7f3a91-4b2e-4f8a-9c1d-2e6b8a0f5d31
  &redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Fcallback
  &scope=mcp%3Aread%20products%3Asearch
  &state=f4k3st8x
  &code_challenge=E9Melhoa2OwvFrEMTJguCHao4K1Cx2VUb7xJ
  &code_challenge_method=S256
  &resource=https%3A%2F%2Fstore.example.com

Open it with webbrowser.open(). The human approves; the AS redirects to your loopback. Capture the code:

import http.server, threading
from urllib.parse import urlparse, parse_qs

captured = {}

class Callback(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        q = parse_qs(urlparse(self.path).query)
        if "code" in q and q.get("state") == ["f4k3st8x"]:
            captured["code"] = q["code"][0]
            body = b"<h1>Authorized. Close this tab.</h1>"
            self.send_response(200)
            self.send_header("Content-Type", "text/html")
            self.send_header("Content-Length", str(len(body)))
            self.end_headers()
            self.wfile.write(body)
        else:
            self.send_response(400)
            self.end_headers()
    def log_message(self, *args): pass

server = http.server.HTTPServer(("127.0.0.1", 8080), Callback)
threading.Thread(target=server.serve_forever, daemon=True).start()
# ... wait on captured["code"], then server.shutdown()

Check state before accepting anything — a mismatched state means the redirect did not come from your request. Then exchange the code:

POST https://example.com/token
Content-Type: application/x-www-form-urlencoded

grant_type=authorization_code
&code=OC_7f2b9e...
&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Fcallback
&client_id=dc7f3a91-4b2e-4f8a-9c1d-2e6b8a0f5d31
&code_verifier=dBjftJeZ4CVP-mB92K27uhbUJU1p1r_wW1gFWFOEjXk
{
  "access_token": "eyJhbGciOiJ9...",
  "token_type": "Bearer",
  "expires_in": 3600,
  "refresh_token": "rt_9f2c41ab",
  "scope": "mcp:read products:search"
}

Attaching the Access Token to MCP JSON-RPC Tool Calls over HTTP

The token rides on every request as a Bearer header. Over streamable HTTP, each POST carries both Accept types, and the sequence is initialize, then notifications/initialized, then tools/call:

POST /mcp HTTP/1.1
Host: store.example.com
Authorization: Bearer eyJhbGciOiJ9...
Content-Type: application/json
Accept: application/json, text/event-stream

{"jsonrpc": "2.0", "id": 1, "method": "initialize",
 "params": {"protocolVersion": "LATEST", "capabilities": {},
            "clientInfo": {"name": "store-mcp-agent", "version": "0.1.0"}}}
{"jsonrpc": "2.0", "method": "notifications/initialized"}
{"jsonrpc": "2.0", "id": 2, "method": "tools/call",
 "params": {"name": "store.search_products",
            "arguments": {"query": "wireless earbuds", "limit": 5}}}
{
  "jsonrpc": "2.0",
  "id": 2,
  "result": {
    "content": [
      {"type": "text", "text": "{\"products\": [{\"title\": \"Wireless Earbuds Pro\", \"price\": 79.99, \"url\": \"https://store.example.com/p/earbuds-pro\"}, {\"title\": \"Earbuds Lite\", \"price\": 34.50, \"url\": \"https://store.example.com/p/earbuds-lite\"}]}"},
      {"type": "text", "text": "2 products matched."}
    ]
  }
}

One architectural point worth pausing on: the token your agent holds is for the MCP server, not for whatever the server does internally. If the tool implementation calls a scraping API — here FineData’s — that key stays server-side and the agent never sees it:

# inside the MCP server's store.search_products tool
resp = requests.post(
    "https://api.finedata.ai/api/v1/async/scrape",
    headers={"Authorization": "Bearer fd_server_side_key"},
    json={
        "url": product_listing_url,  # required: the page to scrape
        "extract_rules": {"title": ".product-title", "price": ".price"},
        "formats": ["text"],
    },
    timeout=120,
)

Audience separation is what makes this safe. A leaked agent token cannot be replayed against the scraping API, and a leaked scraping key never crosses into agent-observable memory.

Intercepting 401s on Tool Calls and Refreshing the Access Token

Access tokens expire mid-session. When one does, the server answers before JSON-RPC ever runs:

HTTP/1.1 401 Unauthorized
Content-Type: application/json
WWW-Authenticate: Bearer error="invalid_token", error_description="access token expired"

{"error": "invalid_token", "error_description": "access token expired"}

Read WWW-Authenticate before reacting. Only error="invalid_token" means “refresh and replay” — other errors mean different things, and blind retries will not fix them. Wrap the transport so every tool call gets the treatment:

import time, requests

class ReauthorizationNeeded(Exception): pass

class StoreMCPClient:
    def __init__(self, mcp_url, token_endpoint, client_id, tokens):
        self.mcp_url = mcp_url
        self.token_endpoint = token_endpoint
        self.client_id = client_id
        self.tokens = tokens   # {"access_token", "refresh_token", "expires_at"}
        self._id = 0

    def _rpc(self, payload):
        return requests.post(
            self.mcp_url,
            json=payload,
            headers={
                "Authorization": f"Bearer {self.tokens['access_token']}",
                "Accept": "application/json, text/event-stream",
            },
            timeout=30,
        )

    def call_tool(self, name, arguments):
        self._id += 1
        payload = {"jsonrpc": "2.0", "id": self._id, "method": "tools/call",
                   "params": {"name": name, "arguments": arguments}}
        resp = self._rpc(payload)
        if resp.status_code == 401 and 'error="invalid_token"' in \
                resp.headers.get("WWW-Authenticate", ""):
            self._refresh()
            resp = self._rpc(payload)   # replay with the new access token
        resp.raise_for_status()
        return resp.json()

    def _refresh(self):
        r = requests.post(
            self.token_endpoint,
            data={"grant_type": "refresh_token",
                  "refresh_token": self.tokens["refresh_token"],
                  "client_id": self.client_id},
            timeout=10,
        )
        if r.status_code == 400 and r.json().get("error") == "invalid_grant":
            raise ReauthorizationNeeded("refresh token dead -- rerun PKCE")
        tok = r.json()
        self.tokens = {"access_token": tok["access_token"],
                       "refresh_token": tok["refresh_token"],
                       "expires_at": time.time() + tok["expires_in"]}

Surviving Refresh Token Rotation and Forced Re-Authorization

Many authorization servers rotate the refresh token on every use: each refresh response carries a new one, and the old one dies immediately. Two concurrent tool calls that both decide to refresh will race — the loser posts an already-used token, gets invalid_grant, and forces a human back into the loop. That is the failure you must engineer away.

SignalMeaningAgent action
401, WWW-Authenticate error="invalid_token"access token expired or revokedsilent refresh, replay the call
400 {"error": "invalid_grant"} on refreshrefresh token rotated, revoked, or expireddrop tokens, restart full PKCE (human consent)
403, error="insufficient_scope"token lacks scope for this toolre-run authorize with the wider scope
401/400 error="invalid_client" at the token endpointclient registration revokedre-register dynamically, then full flow

The fix is a single-flight lock around refresh. One caller refreshes; everyone else waits and reuses the result:

import threading, time

class TokenStore:
    """Single-flight refresh: one lock, held across the network call."""
    def __init__(self, tokens):
        self._lock = threading.Lock()
        self.tokens = tokens

    def access_token(self, refresh_fn):
        with self._lock:
            t = self.tokens
            if t and t["expires_at"] > time.time() + 30:
                return t["access_token"]        # proactive margin
            fresh = refresh_fn(t["refresh_token"])
            self.tokens = fresh                 # rotated pair lands atomically
            return fresh["access_token"]

Yes, this holds a lock across network I/O, which serializes tool calls during a refresh. I do it anyway. A 90 ms serialized refresh beats two racing refreshes where one loses and triggers a full re-consent with a human in the loop. If you disagree and prefer lock-free refresh with compare-and-swap on the token pair, that works too — just make sure the loser re-reads the stored refresh token and retries once before giving up.

Verifying the Full Auth Loop Locally with Timed Curl Steps

Do not trust the loop until you have watched every leg complete. Run a local mock MCP server that also plays the authorization server — it auto-approves consent and mints access tokens with a short expires_in so you can force the 401 path on demand. Then drive it with curl and millisecond timestamps:

#!/usr/bin/env bash
set -euo pipefail
BASE=http://localhost:9000     # local mock: MCP server + AS in one process
now() { date +%s%3N; }

t0=$(now)
AS=$(curl -sf "$BASE/.well-known/oauth-protected-resource" \
  | jq -r '.authorization_servers[0]')
META=$(curl -sf "$AS/.well-known/oauth-authorization-server")
t1=$(now); echo "discovery:        $((t1-t0)) ms"
# validate issuer == $AS here, or stop

CLIENT_ID=$(curl -sf -X POST "$(echo "$META" | jq -r .registration_endpoint)" \
  -H 'Content-Type: application/json' \
  -d '{"client_name":"store-mcp-agent",
       "redirect_uris":["http://localhost:8080/callback"],
       "grant_types":["authorization_code","refresh_token"],
       "token_endpoint_auth_method":"none"}' | jq -r .client_id)
t2=$(now); echo "registration:     $((t2-t1)) ms"

VERIFIER=$(head -c32 /dev/urandom | basenc --base64url | tr -d '=')   # 43 chars
CHALLENGE=$(printf %s "$VERIFIER" | openssl dgst -sha256 -binary \
  | basenc --base64url | tr -d '=')

REDIRECT=$(curl -s -o /dev/null -w '%{redirect_url}' \
  "$(echo "$META" | jq -r .authorization_endpoint)?response_type=code&client_id=$CLIENT_ID&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Fcallback&scope=mcp%3Aread+products%3Asearch&code_challenge=$CHALLENGE&code_challenge_method=S256&resource=http%3A%2F%2Flocalhost%3A9000")
CODE=${REDIRECT##*code=}; CODE=${CODE%%&*}
TOKENS=$(curl -sf -X POST "$(echo "$META" | jq -r .token_endpoint)" \
  --data-urlencode grant_type=authorization_code \
  --data-urlencode "code=$CODE" \
  --data-urlencode "redirect_uri=http://localhost:8080/callback" \
  --data-urlencode "client_id=$CLIENT_ID" \
  --data-urlencode "code_verifier=$VERIFIER")
t3=$(now); echo "pkce round trip:  $((t3-t2)) ms"

AT=$(echo "$TOKENS" | jq -r .access_token)
curl -sf -X POST "$BASE/mcp" -H "Authorization: Bearer $AT" \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call",
       "params":{"name":"store.search_products",
                 "arguments":{"query":"wireless earbuds"}}}' | jq .

sleep 6                        # mock mints expires_in=5
t4=$(now)
CODE401=$(curl -s -o /dev/null -w '%{http_code}' -X POST "$BASE/mcp" \
  -H "Authorization: Bearer $AT" \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call",
       "params":{"name":"store.search_products",
                 "arguments":{"query":"wireless earbuds"}}}')
echo "expired call:   HTTP $CODE401"   # expect 401

AT=$(curl -sf -X POST "$(echo "$META" | jq -r .token_endpoint)" \
  --data-urlencode grant_type=refresh_token \
  --data-urlencode "refresh_token=$(echo "$TOKENS" | jq -r .refresh_token)" \
  --data-urlencode "client_id=$CLIENT_ID" | jq -r .access_token)
t5=$(now); echo "silent refresh:   $((t5-t4)) ms"

curl -sf -X POST "$BASE/mcp" -H "Authorization: Bearer $AT" \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":3,"method":"tools/call",
       "params":{"name":"store.search_products",
                 "arguments":{"query":"wireless earbuds"}}}' | jq .

A sample run on my machine, mock on loopback:

StepSample timing
Metadata discovery (two GETs)~120 ms
Dynamic registration~180 ms
Full PKCE round trip (browser-bound)~2.3 s
Silent refresh~90 ms
Added latency, one 401 → refresh → replay cycle~250 ms

Everything before the first tool call is one-time per session, so do not bother optimizing it. The 401 cycle is the only cost that lands on the hot path — roughly one 401 round trip plus a 90 ms refresh plus the replayed call — and with a 3600-second access token it bites about once per hour. Refresh proactively 30 seconds early and even that disappears.

Wrapping Up

The loop is: discover, register, consent with PKCE, attach, intercept 401s, refresh under a single-flight lock, and re-consent when rotation kills you anyway. Getchas worth taping to the monitor:

  • Issuer mismatch — refuse and stop; do not “try anyway.”
  • HTML instead of JSON — you hit the wrong well-known path or POSTed to the authorize endpoint; the symptoms are distinctive.
  • Missing resource parameter — the token is valid but not audience-bound, so the MCP server 401s a fresh token. Confusing, and entirely avoidable.
  • Token persistence — a refresh token is full agent access; store it with restrictive file permissions, never in logs.
  • Loopback port — RFC 8252 permits any port, but register the exact URI you will serve.

Next steps: read up on auth scopes and failure modes for MCP tools before you widen your agent’s scopes, and if your MCP server fronts a scraping API like FineData’s, keep that key server-side — the audience separation you just built is what keeps the two credential worlds from bleeding into each other.

#mcp #oauth #ai-agents #pkce #authentication #slot:mcp-agent

Related Articles