Web Scraping Tools
Your AI Agent Can Call
The FineData MCP server gives an agent nine tools: read a page, send a POST or DELETE, queue work, submit a batch, poll or cancel a job, and check its own token balance. Pages arrive as markdown by default — the format a language model actually reads — and only successful requests are billed.
Run it locally over stdio with npx or
pip, or connect to the hosted endpoint over Streamable HTTP
and sign in with OAuth instead of pasting a key.
Connect in one config block
Every MCP client reads the same mcpServers object —
Cursor, Claude Desktop, Claude Code, Windsurf, Cline and the rest. Pick the shape that
matches how you run the agent.
Local, with a key
The agent runs the server as a child process. Best for an agent on your own machine.
{
"mcpServers": {
"finedata": {
"command": "npx",
"args": ["-y", "@finedata/mcp-server"],
"env": { "FINEDATA_API_KEY": "fd_your_api_key" }
}
}
}
Python instead of Node: pip install finedata-mcp
Remote, with a key
Nothing to install or update. Best for hosted agents and CI.
{
"mcpServers": {
"finedata": {
"url": "https://mcp.finedata.ai/mcp",
"headers": { "Authorization": "Bearer fd_your_api_key" }
}
}
}Keys come from your dashboard and carry your own plan and balance.
Remote, with sign-in
No credentials in the config at all. Best when someone else runs the agent.
{
"mcpServers": {
"finedata": {
"url": "https://mcp.finedata.ai/mcp"
}
}
}The client registers itself and opens a browser to sign in. See how that works.
The nine tools
Each one is annotated for the agent: whether it only reads, whether repeating it is safe,
and roughly what it costs. An agent that understands those hints stops polling in a loop
and stops paying for retries it did not need. Reading and writing are deliberately
separate tools — scrape_url is always a GET,
so a client can run it without asking, while anything that may change something at the
other end goes through send_http_request.
| Tool | What it does | Scope |
|---|---|---|
| scrape_url | Read one page with GET and return markdown, HTML, links or structured JSON. | scrape:write |
| send_http_request | Send POST, PUT, PATCH or DELETE through the same pipeline, for forms and write APIs. | scrape:write |
| scrape_async | Queue a page and get a job id back, for slow targets or long batches. | scrape:write |
| get_job_status | Poll one queued job and read its result when finished. | jobs:read |
| list_jobs | List recent jobs with their state, to resume after a restart. | jobs:read |
| cancel_job | Stop a queued job that is no longer needed. | scrape:write |
| batch_scrape | Submit many URLs in one call with shared options. | scrape:write |
| get_batch_status | Track a batch and collect per-URL results. | jobs:read |
| get_usage | Read token balance and consumption, so the agent can budget itself. | usage:read |
A call looks like this
{
"url": "https://example.com/product/1",
"formats": ["markdown"],
"stealth_premium": true,
"use_isp": true
}Escalate by cost, not by habit
Start with a plain request. If a page comes back empty or refuses to serve, step up:
stealth_antibot, then
stealth_premium together with
use_isp — that pairing is the strongest common step,
and premium on datacenter routing alone is much weaker.
use_residential and
use_mobile exist for genuinely geo- or IP-sensitive
targets, not as a default. The server returns the next step as a hint, so the agent
escalates on evidence rather than on a guess.
Sign-in instead of a shared key
An API key is a bearer secret: whoever holds it is you. That is fine for an agent you run yourself and awkward for one you hand to a teammate or a customer. The remote endpoint therefore speaks OAuth 2.1, so each person connects their own account.
What happens
- The client discovers the endpoint's metadata and registers itself — no client id to create by hand.
- A browser opens on a consent screen naming the client and the scopes it asks for.
- You sign in to your own account and approve. The client receives a short-lived token, verified with PKCE so an intercepted redirect is useless.
- Requests bill to your account. The token lasts an hour and renews quietly for up to 30 days.
What the token cannot do
Approving "scrape on my behalf" grants exactly that. The token reaches the nine tools and nothing else: it cannot create API keys, change your plan, spend from your wallet, or read your profile — the endpoints behind those actions refuse it outright.
Scopes are enforced, not decorative: a client granted read-only scopes can poll jobs and cannot start them. Revoke a connection from your dashboard at any time; changing your password cuts every connected agent as well as every browser session.
Why cost matters more for agents
Failures are not billed
An agent retrying on its own judgement would otherwise be an open budget. You pay for requests that returned the page.
The agent can see the bill
Every response reports tokens_used, and
get_usage reports the balance — so a loop can stop
itself instead of discovering the cost afterwards.
Markdown by default
Raw HTML burns context for nothing. Long pages are truncated with a note rather than silently cut, so the model knows what it did not see.
Questions
What is the FineData MCP server?
It is a Model Context Protocol server that gives an AI agent nine web-scraping tools: read a page, send a POST or DELETE, queue work asynchronously, submit batches, poll or cancel jobs, and read token usage. It runs locally over stdio via npx or pip, or remotely over Streamable HTTP at https://mcp.finedata.ai/mcp. Pages come back as markdown by default, which is what a language model reads best.
How do I connect FineData MCP to Cursor or Claude Desktop?
Add one entry to the client's mcpServers config. Locally: command "npx" with args ["-y", "@finedata/mcp-server"] and FINEDATA_API_KEY in env. Remotely: url "https://mcp.finedata.ai/mcp" with an Authorization: Bearer header, or no credentials at all if the client supports OAuth, in which case the client registers itself and opens a browser sign-in.
Do I need an API key, or does OAuth replace it?
Either works. An API key is simplest for a server-side agent you control. OAuth suits a client you hand to someone else: the user signs in to their own FineData account, approves the scopes on a consent screen, and usage bills to their account rather than yours. Nothing is copied or pasted, and the connection can be revoked later.
What can an agent do with an OAuth connection to my account?
Only the nine tools, and only what the approved scopes cover: scrape:write to fetch and cancel, jobs:read to poll, usage:read to check the balance. The token is confined to those endpoints, so it cannot create API keys, change a plan, touch billing or read profile data. It expires after an hour, refreshes for up to 30 days, and is revoked by a password change or from the dashboard.
How much does an MCP scrape cost?
Tokens per successful request, and failed requests are not billed. A plain fetch is about 3 tokens; heavier rendering and residential or ISP routing cost more, and the tool descriptions carry the current per-mode figures so the agent can pick the cheapest option that works. The response reports tokens_used, so an agent can measure instead of guessing.
Which modes should an agent try first?
Cheapest first. Start with a plain request; if the page comes back empty or blocked, retry with stealth_antibot, then with stealth_premium plus use_isp, which is the strongest common step. Reserve use_residential and use_mobile for targets that are genuinely geo or IP sensitive rather than using them by default. The server returns the next step as a hint, so the agent does not have to hardcode the ladder.
Is the MCP server open source, and where is it published?
The server is published as finedata-mcp on PyPI and @finedata/mcp-server on npm, and is listed in the MCP Registry. The remote endpoint publishes OAuth Authorization Server Metadata and Protected Resource Metadata, so a compliant client can discover and connect to it without any hand-written configuration.
Agents act quickly and at volume, which makes the rules matter more, not less. Collect only what you have a lawful basis to collect, respect each site's terms and robots directives, and keep request rates civil. Our Acceptable Use Policy sets the boundaries, and we make no representation that scraping any particular site is permissible.
Related Guides
MCP: how agents get live web data
What the protocol standardises, and why a tool interface beats a hand-rolled HTTP wrapper.
Where scraping is going as LLMs consume it
Why the output format and the cost signal matter once a model, not a script, is the caller.
Scraping API vs building it yourself
The cost of maintaining browsers, proxies and retries against paying per successful request.