Authorize an AI Agent to a Remote MCP Server
Wire OAuth 2.1 into an MCP agent client: discover the server's auth metadata, register dynamically, run PKCE, and refresh tokens when tool calls 401.
On this page
Authorize an AI Agent to a Remote MCP Server
Your agent just tried its first tool call against a remote MCP server and got back a 401. Locally, over stdio, the same server worked fine — because a local server inherits your operating system trust and never asks who is calling. Over HTTP, that free ride ends. Every request needs a Bearer token, and something has to obtain, attach, refresh, and re-attach that token on the agent’s behalf.
That something is your client code. MCP delegates remote-server auth to OAuth 2.1: metadata discovery, dynamic client registration, authorization code + PKCE, and refresh handling. This tutorial wires the full loop into a Python agent client, then proves it works with a timed end-to-end run.
Where OAuth 2.1 Fits: Local stdio MCP vs. a Remote HTTP MCP Server
The difference between the two transports is trust, not features. A stdio server is a child process running as the same OS user as the agent — it reads the same environment and files, so there is no boundary to authenticate across. A remote server at store.example.com sits on a public network, trusts nothing by default, and demands a scoped, audience-bound Bearer token on every request.
| Local stdio MCP | Remote HTTP MCP (store.example.com) | |
|---|---|---|
| Transport | stdin/stdout, child process | POST /mcp over HTTPS |
| Trust boundary | none — same OS user as the agent | public network; server trusts nothing by default |
| Credential scope | anything the launching user can read | one Bearer token per request, scoped and audience-bound |
| Auth story | inherited OS permissions | OAuth 2.1: discovery, PKCE, refresh |
Four actors participate. The agent is the OAuth client. The MCP server is the resource. The authorization server at example.com issues tokens. A human — the resource owner — approves consent in a browser. Even a headless agent needs the human for that one step; after it, the loop runs on its own.
Agent (client) MCP server Authorization server Human
store.example.com example.com (browser)
|-- GET /.well-known/oauth-protected-resource -->|
|<-- { resource, authorization_servers } ---------|
|-- GET /.well-known/oauth-authorization-server --------->|
|<-- { issuer, endpoints, scopes } ------------------------|
|-- POST /register ---------------------------------------->|
|<-- client_id --------------------------------------------|
|-- GET /authorize?...&code_challenge=... ---------------->|--> consent
| |--> approve
|<== redirect to http://localhost:8080/callback?code=... ===|<== browser
|-- POST /token { code, code_verifier } ------------------>|
|<-- access_token, refresh_token --------------------------|
|-- POST /mcp Authorization: Bearer ... -->|
|<-- JSON-RPC tool result ------------------|
Discovering the Authorization Server via /.well-known/oauth-protected-resource
Do not hardcode endpoint URLs. The MCP spec points clients at the protected-resource document, which names the authorization server, which in turn publishes its own endpoints. Two GETs and you know everything.
GET https://store.example.com/.well-known/oauth-protected-resource
{
"resource": "https://store.example.com",
"authorization_servers": ["https://example.com"],
"scopes_supported": ["mcp:read", "products:search"]
}
The authorization_servers entry is your hop two. Fetch the AS metadata document from that origin:
GET https://example.com/.well-known/oauth-authorization-server
{
"issuer": "https://example.com",
"authorization_endpoint": "https://example.com/authorize",
"token_endpoint": "https://example.com/token",
"registration_endpoint": "https://example.com/register",
"scopes_supported": ["mcp:read", "products:search"],
"response_types_supported": ["code"],
"grant_types_supported": ["authorization_code", "refresh_token"],
"code_challenge_methods_supported": ["S256"],
"token_endpoint_auth_methods_supported": ["none"]
}
One validation matters more than the rest: issuer must match the URL you actually fetched, after normalizing trailing slashes. A mismatch means DNS tricks or a path-prefixed deployment, and tokens minted there will not be audience-bound to your MCP server. Refuse and stop.
import requests
def discover(store_url: str) -> dict:
"""Two-step discovery: protected-resource doc, then AS metadata."""
pr = requests.get(
f"{store_url}/.well-known/oauth-protected-resource", timeout=10
)
pr.raise_for_status()
protected = pr.json()
as_url = protected["authorization_servers"][0].rstrip("/")
meta = requests.get(
f"{as_url}/.well-known/oauth-authorization-server", timeout=10
)
meta.raise_for_status()
auth_server = meta.json()
if auth_server["issuer"].rstrip("/") != as_url:
raise ValueError(
f"issuer mismatch: {auth_server['issuer']} != {as_url}"
)
return {
"resource": protected["resource"],
"issuer": as_url,
"authorize": auth_server["authorization_endpoint"],
"token": auth_server["token_endpoint"],
"register": auth_server["registration_endpoint"],
"scopes": auth_server["scopes_supported"],
}
I re-run this every session rather than caching metadata to disk. It costs about 120 ms, and stale endpoint URLs are exactly the bug you want to catch at startup instead of mid-session. If you point your client at the wrong well-known path you get an HTML 404 page back and a JSONDecodeError downstream — that failure mode has its own post.
Registering the Agent Dynamically at the Authorization Server’s /register Endpoint
Dynamic client registration (RFC 7591) removes the dev-console step: the agent POSTs its own metadata to the AS and receives a client_id on the spot. No pre-provisioned credentials, no ticket to file.
The agent is a public client — it cannot hold a secret, because anything compiled into or read by the agent process is effectively public. So it registers with token_endpoint_auth_method: "none" and lets PKCE compensate. The redirect URI is loopback, per RFC 8252.
{
"client_name": "store-mcp-agent",
"redirect_uris": ["http://localhost:8080/callback"],
"grant_types": ["authorization_code", "refresh_token"],
"response_types": ["code"],
"token_endpoint_auth_method": "none",
"scope": "mcp:read products:search"
}
{
"client_id": "dc7f3a91-4b2e-4f8a-9c1d-2e6b8a0f5d31",
"redirect_uris": ["http://localhost:8080/callback"],
"grant_types": ["authorization_code", "refresh_token"],
"response_types": ["code"],
"token_endpoint_auth_method": "none"
}
Persist the registration next to the discovered endpoints, because you will need client_id on every refresh:
import json, pathlib, requests
def register(as_meta: dict) -> dict:
resp = requests.post(
as_meta["register"],
json={
"client_name": "store-mcp-agent",
"redirect_uris": ["http://localhost:8080/callback"],
"grant_types": ["authorization_code", "refresh_token"],
"response_types": ["code"],
"token_endpoint_auth_method": "none",
},
timeout=10,
)
resp.raise_for_status()
reg = resp.json()
state = pathlib.Path("~/.store-mcp-agent/registration.json").expanduser()
state.parent.mkdir(parents=True, exist_ok=True)
state.write_text(json.dumps(reg, indent=2))
return reg
Two caveats. If registration_endpoint is absent from the AS metadata, that server does not support dynamic registration — fall back to a manually issued client_id and skip this step. And register the exact loopback URI you will serve; the AS matches redirect URIs byte-for-byte.
Running the Authorization Code + PKCE Flow from a Headless Agent
PKCE binds the authorization request to the token exchange, so a intercepted authorization code is useless without the verifier. Generate both with secrets:
import base64, hashlib, secrets
code_verifier = base64.urlsafe_b64encode(
secrets.token_bytes(32)
).rstrip(b"=").decode()
assert len(code_verifier) == 43
code_challenge = base64.urlsafe_b64encode(
hashlib.sha256(code_verifier.encode()).digest()
).rstrip(b"=").decode()
Fully rendered, the consent URL looks like this. Note the resource parameter (RFC 8707) — it tells the AS to mint an audience-bound token for store.example.com, which the MCP server will check. Omit it and you get a token that is technically valid but rejected by the server:
https://example.com/authorize
?response_type=code
&client_id=dc7f3a91-4b2e-4f8a-9c1d-2e6b8a0f5d31
&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Fcallback
&scope=mcp%3Aread%20products%3Asearch
&state=f4k3st8x
&code_challenge=E9Melhoa2OwvFrEMTJguCHao4K1Cx2VUb7xJ
&code_challenge_method=S256
&resource=https%3A%2F%2Fstore.example.com
Open it with webbrowser.open(). The human approves; the AS redirects to your loopback. Capture the code:
import http.server, threading
from urllib.parse import urlparse, parse_qs
captured = {}
class Callback(http.server.BaseHTTPRequestHandler):
def do_GET(self):
q = parse_qs(urlparse(self.path).query)
if "code" in q and q.get("state") == ["f4k3st8x"]:
captured["code"] = q["code"][0]
body = b"<h1>Authorized. Close this tab.</h1>"
self.send_response(200)
self.send_header("Content-Type", "text/html")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
else:
self.send_response(400)
self.end_headers()
def log_message(self, *args): pass
server = http.server.HTTPServer(("127.0.0.1", 8080), Callback)
threading.Thread(target=server.serve_forever, daemon=True).start()
# ... wait on captured["code"], then server.shutdown()
Check state before accepting anything — a mismatched state means the redirect did not come from your request. Then exchange the code:
POST https://example.com/token
Content-Type: application/x-www-form-urlencoded
grant_type=authorization_code
&code=OC_7f2b9e...
&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Fcallback
&client_id=dc7f3a91-4b2e-4f8a-9c1d-2e6b8a0f5d31
&code_verifier=dBjftJeZ4CVP-mB92K27uhbUJU1p1r_wW1gFWFOEjXk
{
"access_token": "eyJhbGciOiJ9...",
"token_type": "Bearer",
"expires_in": 3600,
"refresh_token": "rt_9f2c41ab",
"scope": "mcp:read products:search"
}
Attaching the Access Token to MCP JSON-RPC Tool Calls over HTTP
The token rides on every request as a Bearer header. Over streamable HTTP, each POST carries both Accept types, and the sequence is initialize, then notifications/initialized, then tools/call:
POST /mcp HTTP/1.1
Host: store.example.com
Authorization: Bearer eyJhbGciOiJ9...
Content-Type: application/json
Accept: application/json, text/event-stream
{"jsonrpc": "2.0", "id": 1, "method": "initialize",
"params": {"protocolVersion": "LATEST", "capabilities": {},
"clientInfo": {"name": "store-mcp-agent", "version": "0.1.0"}}}
{"jsonrpc": "2.0", "method": "notifications/initialized"}
{"jsonrpc": "2.0", "id": 2, "method": "tools/call",
"params": {"name": "store.search_products",
"arguments": {"query": "wireless earbuds", "limit": 5}}}
{
"jsonrpc": "2.0",
"id": 2,
"result": {
"content": [
{"type": "text", "text": "{\"products\": [{\"title\": \"Wireless Earbuds Pro\", \"price\": 79.99, \"url\": \"https://store.example.com/p/earbuds-pro\"}, {\"title\": \"Earbuds Lite\", \"price\": 34.50, \"url\": \"https://store.example.com/p/earbuds-lite\"}]}"},
{"type": "text", "text": "2 products matched."}
]
}
}
One architectural point worth pausing on: the token your agent holds is for the MCP server, not for whatever the server does internally. If the tool implementation calls a scraping API — here FineData’s — that key stays server-side and the agent never sees it:
# inside the MCP server's store.search_products tool
resp = requests.post(
"https://api.finedata.ai/api/v1/async/scrape",
headers={"Authorization": "Bearer fd_server_side_key"},
json={
"url": product_listing_url, # required: the page to scrape
"extract_rules": {"title": ".product-title", "price": ".price"},
"formats": ["text"],
},
timeout=120,
)
Audience separation is what makes this safe. A leaked agent token cannot be replayed against the scraping API, and a leaked scraping key never crosses into agent-observable memory.
Intercepting 401s on Tool Calls and Refreshing the Access Token
Access tokens expire mid-session. When one does, the server answers before JSON-RPC ever runs:
HTTP/1.1 401 Unauthorized
Content-Type: application/json
WWW-Authenticate: Bearer error="invalid_token", error_description="access token expired"
{"error": "invalid_token", "error_description": "access token expired"}
Read WWW-Authenticate before reacting. Only error="invalid_token" means “refresh and replay” — other errors mean different things, and blind retries will not fix them. Wrap the transport so every tool call gets the treatment:
import time, requests
class ReauthorizationNeeded(Exception): pass
class StoreMCPClient:
def __init__(self, mcp_url, token_endpoint, client_id, tokens):
self.mcp_url = mcp_url
self.token_endpoint = token_endpoint
self.client_id = client_id
self.tokens = tokens # {"access_token", "refresh_token", "expires_at"}
self._id = 0
def _rpc(self, payload):
return requests.post(
self.mcp_url,
json=payload,
headers={
"Authorization": f"Bearer {self.tokens['access_token']}",
"Accept": "application/json, text/event-stream",
},
timeout=30,
)
def call_tool(self, name, arguments):
self._id += 1
payload = {"jsonrpc": "2.0", "id": self._id, "method": "tools/call",
"params": {"name": name, "arguments": arguments}}
resp = self._rpc(payload)
if resp.status_code == 401 and 'error="invalid_token"' in \
resp.headers.get("WWW-Authenticate", ""):
self._refresh()
resp = self._rpc(payload) # replay with the new access token
resp.raise_for_status()
return resp.json()
def _refresh(self):
r = requests.post(
self.token_endpoint,
data={"grant_type": "refresh_token",
"refresh_token": self.tokens["refresh_token"],
"client_id": self.client_id},
timeout=10,
)
if r.status_code == 400 and r.json().get("error") == "invalid_grant":
raise ReauthorizationNeeded("refresh token dead -- rerun PKCE")
tok = r.json()
self.tokens = {"access_token": tok["access_token"],
"refresh_token": tok["refresh_token"],
"expires_at": time.time() + tok["expires_in"]}
Surviving Refresh Token Rotation and Forced Re-Authorization
Many authorization servers rotate the refresh token on every use: each refresh response carries a new one, and the old one dies immediately. Two concurrent tool calls that both decide to refresh will race — the loser posts an already-used token, gets invalid_grant, and forces a human back into the loop. That is the failure you must engineer away.
| Signal | Meaning | Agent action |
|---|---|---|
401, WWW-Authenticate error="invalid_token" | access token expired or revoked | silent refresh, replay the call |
400 {"error": "invalid_grant"} on refresh | refresh token rotated, revoked, or expired | drop tokens, restart full PKCE (human consent) |
403, error="insufficient_scope" | token lacks scope for this tool | re-run authorize with the wider scope |
401/400 error="invalid_client" at the token endpoint | client registration revoked | re-register dynamically, then full flow |
The fix is a single-flight lock around refresh. One caller refreshes; everyone else waits and reuses the result:
import threading, time
class TokenStore:
"""Single-flight refresh: one lock, held across the network call."""
def __init__(self, tokens):
self._lock = threading.Lock()
self.tokens = tokens
def access_token(self, refresh_fn):
with self._lock:
t = self.tokens
if t and t["expires_at"] > time.time() + 30:
return t["access_token"] # proactive margin
fresh = refresh_fn(t["refresh_token"])
self.tokens = fresh # rotated pair lands atomically
return fresh["access_token"]
Yes, this holds a lock across network I/O, which serializes tool calls during a refresh. I do it anyway. A 90 ms serialized refresh beats two racing refreshes where one loses and triggers a full re-consent with a human in the loop. If you disagree and prefer lock-free refresh with compare-and-swap on the token pair, that works too — just make sure the loser re-reads the stored refresh token and retries once before giving up.
Verifying the Full Auth Loop Locally with Timed Curl Steps
Do not trust the loop until you have watched every leg complete. Run a local mock MCP server that also plays the authorization server — it auto-approves consent and mints access tokens with a short expires_in so you can force the 401 path on demand. Then drive it with curl and millisecond timestamps:
#!/usr/bin/env bash
set -euo pipefail
BASE=http://localhost:9000 # local mock: MCP server + AS in one process
now() { date +%s%3N; }
t0=$(now)
AS=$(curl -sf "$BASE/.well-known/oauth-protected-resource" \
| jq -r '.authorization_servers[0]')
META=$(curl -sf "$AS/.well-known/oauth-authorization-server")
t1=$(now); echo "discovery: $((t1-t0)) ms"
# validate issuer == $AS here, or stop
CLIENT_ID=$(curl -sf -X POST "$(echo "$META" | jq -r .registration_endpoint)" \
-H 'Content-Type: application/json' \
-d '{"client_name":"store-mcp-agent",
"redirect_uris":["http://localhost:8080/callback"],
"grant_types":["authorization_code","refresh_token"],
"token_endpoint_auth_method":"none"}' | jq -r .client_id)
t2=$(now); echo "registration: $((t2-t1)) ms"
VERIFIER=$(head -c32 /dev/urandom | basenc --base64url | tr -d '=') # 43 chars
CHALLENGE=$(printf %s "$VERIFIER" | openssl dgst -sha256 -binary \
| basenc --base64url | tr -d '=')
REDIRECT=$(curl -s -o /dev/null -w '%{redirect_url}' \
"$(echo "$META" | jq -r .authorization_endpoint)?response_type=code&client_id=$CLIENT_ID&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Fcallback&scope=mcp%3Aread+products%3Asearch&code_challenge=$CHALLENGE&code_challenge_method=S256&resource=http%3A%2F%2Flocalhost%3A9000")
CODE=${REDIRECT##*code=}; CODE=${CODE%%&*}
TOKENS=$(curl -sf -X POST "$(echo "$META" | jq -r .token_endpoint)" \
--data-urlencode grant_type=authorization_code \
--data-urlencode "code=$CODE" \
--data-urlencode "redirect_uri=http://localhost:8080/callback" \
--data-urlencode "client_id=$CLIENT_ID" \
--data-urlencode "code_verifier=$VERIFIER")
t3=$(now); echo "pkce round trip: $((t3-t2)) ms"
AT=$(echo "$TOKENS" | jq -r .access_token)
curl -sf -X POST "$BASE/mcp" -H "Authorization: Bearer $AT" \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/call",
"params":{"name":"store.search_products",
"arguments":{"query":"wireless earbuds"}}}' | jq .
sleep 6 # mock mints expires_in=5
t4=$(now)
CODE401=$(curl -s -o /dev/null -w '%{http_code}' -X POST "$BASE/mcp" \
-H "Authorization: Bearer $AT" \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/call",
"params":{"name":"store.search_products",
"arguments":{"query":"wireless earbuds"}}}')
echo "expired call: HTTP $CODE401" # expect 401
AT=$(curl -sf -X POST "$(echo "$META" | jq -r .token_endpoint)" \
--data-urlencode grant_type=refresh_token \
--data-urlencode "refresh_token=$(echo "$TOKENS" | jq -r .refresh_token)" \
--data-urlencode "client_id=$CLIENT_ID" | jq -r .access_token)
t5=$(now); echo "silent refresh: $((t5-t4)) ms"
curl -sf -X POST "$BASE/mcp" -H "Authorization: Bearer $AT" \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-d '{"jsonrpc":"2.0","id":3,"method":"tools/call",
"params":{"name":"store.search_products",
"arguments":{"query":"wireless earbuds"}}}' | jq .
A sample run on my machine, mock on loopback:
| Step | Sample timing |
|---|---|
| Metadata discovery (two GETs) | ~120 ms |
| Dynamic registration | ~180 ms |
| Full PKCE round trip (browser-bound) | ~2.3 s |
| Silent refresh | ~90 ms |
| Added latency, one 401 → refresh → replay cycle | ~250 ms |
Everything before the first tool call is one-time per session, so do not bother optimizing it. The 401 cycle is the only cost that lands on the hot path — roughly one 401 round trip plus a 90 ms refresh plus the replayed call — and with a 3600-second access token it bites about once per hour. Refresh proactively 30 seconds early and even that disappears.
Wrapping Up
The loop is: discover, register, consent with PKCE, attach, intercept 401s, refresh under a single-flight lock, and re-consent when rotation kills you anyway. Getchas worth taping to the monitor:
- Issuer mismatch — refuse and stop; do not “try anyway.”
- HTML instead of JSON — you hit the wrong well-known path or POSTed to the authorize endpoint; the symptoms are distinctive.
- Missing
resourceparameter — the token is valid but not audience-bound, so the MCP server 401s a fresh token. Confusing, and entirely avoidable. - Token persistence — a refresh token is full agent access; store it with restrictive file permissions, never in logs.
- Loopback port — RFC 8252 permits any port, but register the exact URI you will serve.
Next steps: read up on auth scopes and failure modes for MCP tools before you widen your agent’s scopes, and if your MCP server fronts a scraping API like FineData’s, keep that key server-side — the audience separation you just built is what keeps the two credential worlds from bleeding into each other.
Related Articles
Confirm Before You Fetch: MCP Elicitation for Scraping Tools
Add MCP elicitation to scraping tools: pause for user consent on costly fetches, collect credentials mid-run, and fail cleanly when no one answers.
TechnicalRun One Web Data Agent Across Multiple MCP Servers
Wire one agent to fetch, extraction, and storage MCP servers. Handle tool name collisions, per-server auth scopes, and wrong-tool picks in production.
TechnicalFence In Your Agent: Domain Allowlists for MCP Fetch Tools
Agents with fetch tools can wander anywhere. Build a domain allowlist into your MCP server, block internal IPs, and log every rejected call.