Batch Requests vs Single Calls: Scraping Pattern Trade-offs

Anatomy of the Two Patterns: Sequential Fetch vs Batched Payload on store.example.com

Every scraping architecture decision eventually lands on the same fork: do you fetch items one at a time, or do you push a list of items into a single request? The two patterns look almost interchangeable on a whiteboard. They are not. They differ in transport shape, failure semantics, retry logic, and — under success-based billing — in what a failed attempt actually costs you.

Here is the sequential pattern, the one most of us write first. One URL, one request, one response:

import requests

PRODUCT_IDS = [101, 102, 103, 104, 105]  # imagine 1,000 of these

session = requests.Session()
session.headers.update({"User-Agent": "my-catalog-sync/1.0"})

results = {}
for pid in PRODUCT_IDS:
    resp = session.get(f"https://store.example.com/products/{pid}", timeout=15)
    if resp.status_code == 200:
        results[pid] = resp.json()
    else:
        # handle it, log it, retry it -- this item, right now
        print(f"product {pid} failed: HTTP {resp.status_code}")

Now the same job as a batched payload. One request carries 50 IDs to a bulk endpoint:

import requests

def fetch_batch(ids):
    resp = requests.post(
        "https://store.example.com/api/products/batch",
        json={"ids": ids},
        headers={"User-Agent": "my-catalog-sync/1.0"},
        timeout=60,
    )
    resp.raise_for_status()
    return resp.json()

results = {}
for i in range(0, len(PRODUCT_IDS), 50):
    chunk = PRODUCT_IDS[i:i + 50]
    batch_resp = fetch_batch(chunk)
    for item in batch_resp["items"]:
        results[item["id"]] = item

Dimension	Sequential (single calls)	Batched payload
Request count for 1,000 IDs	1,000	20 (at 50 IDs per call)
Payload size per request	Tiny (~300 bytes)	Moderate (~2–5 KB up, much larger down)
Endpoint type	Public product page or item API	Bulk/internal-style API endpoint
Failure granularity	Per item — one 404 hurts one ID	Per request — one bad ID can hurt 50
Retry unit	The single failed URL	The failed ID inside a 200 response, or the whole batch
Time to first useful data	One round trip	One round trip per batch

The important column is failure granularity, and we will spend most of this article on it. Throughput differences are real but smaller than people expect. Failure semantics are where the patterns genuinely diverge.

Throughput Math: Why 50 IDs per Call Is Not 50x Faster

The naive assumption is that 50 IDs per request means 50x throughput. In practice you are amortizing connection setup and per-request overhead, and nothing else. The server still has to fetch, serialize, and return 50 items of data.

Here is an instrumented benchmark harness you can run against your own target. I’m using illustrative numbers from a synthetic store.example.com-like service below; your mileage depends entirely on the target’s server-side queue behavior:

import time
import requests

def bench_sequential(ids, session):
    start = time.perf_counter()
    for pid in ids:
        session.get(f"https://store.example.com/products/{pid}", timeout=15)
    return time.perf_counter() - start

def bench_batched(ids, session, batch_size):
    start = time.perf_counter()
    for i in range(0, len(ids), batch_size):
        session.post(
            "https://store.example.com/api/products/batch",
            json={"ids": ids[i:i + batch_size]},
            timeout=60,
        )
    return time.perf_counter() - start

Results for 1,000 product IDs, single-threaded client, same network conditions:

Batch size	Requests	Wall-clock time	Effective per-item time	Speedup vs sequential
1 (sequential)	1,000	500 s	500 ms	1.0x
10	100	95 s	95 ms	5.3x
25	40	55 s	55 ms	9.1x
50	20	38 s	38 ms	13.2x
100	10	32 s	32 ms	15.6x

Notice the curve. Going from 1 to 10 items per call buys you a 5x gain — that is mostly TCP handshake, TLS negotiation, and HTTP overhead disappearing. Going from 50 to 100 buys you almost nothing, because at that point the bottleneck has moved. The server is spending most of its response time queuing and assembling your 100 items, and your client is spending it deserializing a fat JSON blob.

The knee of the curve is where network latency stops dominating and server queue time takes over. For most targets, that knee lands somewhere between 25 and 100 items per call. Past it, bigger batches buy marginal speedup while dramatically increasing your exposure to partial-failure handling, which we cover next.

One more thing the table hides: memory. A 100-item batch response can be several megabytes. Under memory pressure or with connection pooling across many workers, fat batches cause problems that sequential calls never produce.

Retry Logic Diverges: Partial Failures and Per-Item Status Codes

Sequential retries are trivial. The request failed; retry the request. The unit of failure equals the unit of work. Batching breaks that equivalence.

Here is a batch response from a well-behaved bulk endpoint — HTTP 200 at the transport layer, with per-item statuses inside:

{
  "items": [
    {"id": 101, "status": "ok", "data": {"title": "Widget A", "price": 19.99}},
    {"id": 102, "status": "not_found"},
    {"id": 103, "status": "ok", "data": {"title": "Widget C", "price": 24.50}},
    {"id": 104, "status": "rate_limited"},
    {"id": 105, "status": "ok", "data": {"title": "Widget E", "price": 9.99}}
  ]
}

Three of five items succeeded. If you treat this as “batch failed, retry all 50,” you re-fetch 30 items you already have and — depending on billing — may pay for them twice. The correct retry loop parses the per-item statuses and re-queues only the failures:

import time

RETRYABLE = {"rate_limited", "timeout", "server_error"}

def retry_failed(batch_resp, ids, fetch_batch, max_attempts=4):
    failed = [
        item["id"] for item in batch_resp["items"]
        if item["status"] in RETRYABLE
    ]
    attempt = 0
    while failed and attempt < max_attempts:
        backoff = min(2 ** attempt, 60)
        time.sleep(backoff)
        retry_resp = fetch_batch(failed)
        still_failed = [
            item["id"] for item in retry_resp["items"]
            if item["status"] in RETRYABLE
        ]
        failed = still_failed
        attempt += 1
    return failed  # whatever is left needs a dead-letter queue, not more retries

Note the not_found exclusion. A 404-item is not retryable — hammering it wastes budget and annoys the target. Classify before you retry. This is the same discipline described in client-side vs service-Side retries for scraping failures, where the retry loop’s blind spots matter more than its happy path.

Dimension	Sequential	Batched
Retry unit	Whole request (one item)	Single item inside a 200, or whole batch on transport failure
Retry cost	One item’s worth of work	Re-fetch risk for the whole chunk if you retry blindly
Idempotency requirement	Low — GET on one URL	High — must dedupe against already-received items
Dead-letter granularity	One URL per failure entry	Item ID + parent batch ID per failure entry
Backoff strategy	Per-request, simple	Per-item, must respect per-item status semantics

There is also a hybrid failure mode people miss: the transport request itself fails with a 500 after the server partially processed your batch. Did items 1–30 get written to the target’s cache? Unknown. Your retry must be idempotent from your side — track received item IDs, not just completed requests.

The Billing Twist: Paying Only for Successful Requests Reverses the Cost Ranking

This is the part that surprises people. Under metered billing — you pay per request fired — batching looks cheaper because fewer requests means fewer billable events. Under success-based billing, where you pay only for requests that return usable data, the ranking flips in specific, predictable ways.

The mechanics are covered in detail in success-based vs metered scraping API billing, but the short version: with success-based pricing, a failed single call costs you nothing but time. A failed batch call can waste the work of every item in it, depending on whether the provider bills per item or per request.

Monthly cost model for 100,000 product fetches per day (~3M per month), assuming billing counts only successful requests, per-item billing at $0.001 per successful item, and a batch endpoint with all-or-nothing success semantics at $0.05 per successful 50-item batch:

Failure rate	Sequential (per-item)	Batch, per-item billing	Batch, all-or-nothing billing
5%	$2,850	$2,850	$3,000 × (success prob) ≈ $2,700
15%	$2,550	$2,550	≈ $1,700–$2,800 (high variance)
30%	$2,100	$2,100	≈ $700–$2,900 (extreme variance)

Read that last column carefully. Under all-or-nothing semantics, a batch either fully succeeds and bills $0.05 for 50 items, or fully fails and bills nothing. At a 30% per-item failure rate, whole-batch success becomes rare — but when a batch does succeed, you got 50 items for one billable unit. The expected value can be lower, but the variance is brutal. Some days you pay $700. Some days $2,900 for the same work.

Worked example, 100,000 fetches/day, 15% per-item failure rate, independent failures:

• Sequential, success-billed per item: 85,000 successful fetches × $0.001 = $85/day. Predictable. Boring. Every failure is free.
• Batched at 50, all-or-nothing: a batch succeeds only if all 50 items succeed — 0.85^50 ≈ 0.03%. Almost every batch fails, you collect almost no data, and you pay almost nothing. Total daily cost: near zero, daily data collected: near zero. This is the trap.

That second scenario is the whole argument in miniature. All-or-nothing batching under high per-item failure rates doesn’t save money — it destroys your data yield while looking cheap on the invoice. If your target has flaky items (delisted products, geo-restricted SKUs, intermittent 429s), either batch with per-item billing semantics or don’t batch at all.

A quick simulation to see the variance yourself:

import random

def simulate(n_runs=200, n_batches=2000, batch_size=50,
             item_fail_rate=0.15, price_item=0.001, price_batch=0.05):
    per_item_costs, all_or_nothing_costs = [], []
    for _ in range(n_runs):
        # per-item billing: pay for each success
        successes = sum(
            1 for _ in range(n_batches * batch_size)
            if random.random() > item_fail_rate
        )
        per_item_costs.append(successes * price_item)

        # all-or-nothing: pay only when every item in the batch succeeds
        p_all = (1 - item_fail_rate) ** batch_size
        winning_batches = sum(
            1 for _ in range(n_batches) if random.random() < p_all
        )
        all_or_nothing_costs.append(winning_batches * price_batch)

    print(f"per-item:       mean ${sum(per_item_costs)/n_runs:.2f}")
    print(f"all-or-nothing: mean ${sum(all_or_nothing_costs)/n_runs:.2f}")

simulate()

Run it. The per-item mean is stable across runs; the all-or-nothing mean swings wildly. When a cost model has that much variance, you cannot budget against it, and unbudgetable pipelines get killed in the first quarterly review.

Failure Isolation vs Failure Amplification: Choosing by Error Profile

Sequential calls have a property I undervalued for years: failures are contained. One bad ID produces one error. The other 999 fetches proceed untouched.

Batching amplifies. Walk through the scenario: you submit a 50-item batch to store.example.com/api/products/batch, and one ID in the list is malformed — say a stale ID with a non-numeric character from a bad upstream join. If the endpoint validates the payload as a whole, you get a 400 for all 50 items. Forty-nine perfectly good IDs just failed because of one bad one. Your retry loop now has to figure out which item poisoned the chunk, usually by bisecting the batch — extra requests, extra latency, extra complexity that the sequential pattern never needed.

Error type	Sequential blast radius	Batched blast radius
Timeout on one item	One item delayed/retried	Whole chunk retried; per-item progress lost
429 rate limit	One request backed off	Whole chunk backed off; possibly flagged for bulk behavior
Invalid item ID	One 404, logged and skipped	Potential 400 for all 50 items in the chunk
Parser error on one response	One item quarantined	One malformed field can break deserialization of the entire batch payload

The fix for the invalid-item case is pre-validation. Strip anything that cannot possibly be a valid ID before it reaches the batch endpoint:

import re

VALID_ID = re.compile(r"^\d{1,8}$")

def prevalidate(ids, known_bad=frozenset()):
    clean, rejected = [], []
    for pid in ids:
        pid_str = str(pid)
        if not VALID_ID.match(pid_str):
            rejected.append(pid)
        elif pid_str in known_bad:
            rejected.append(pid)
        else:
            clean.append(pid)
    return clean, rejected

clean_chunk, rejected = prevalidate(chunk, known_bad=DEAD_IDS)
if len(clean_chunk) >= 10:
    fetch_batch(clean_chunk)
else:
    # too few survivors -- fall back to sequential for this chunk
    for pid in clean_chunk:
        fetch_single(pid)

The known_bad set matters. Every not_found you observe should go into a persistent dead-ID store so you stop paying round trips for items that will never come back. This is the same refresh-strategy logic discussed in re-scrape vs change detection — knowing what not to fetch is half the cost battle.

My rule of thumb: error-tolerant jobs (catalog syncs, price aggregation, SERP tracking) tolerate batching because a missing item today is fine tomorrow. Error-intolerant jobs — compliance checks, stock verification before a purchase decision, anything with a deadline — should use single calls, because you need to know exactly which item failed and why, right now.

Rate Limits and Politeness: How Batch Shape Affects Your Reputation with the Target

Rate limiting almost always counts requests, not items. Twenty batched requests carrying 1,000 items is a 20-request footprint. One thousand sequential calls is a 1,000-request footprint against the same quota. From a pure quota-consumption view, batching is strictly better.

But reputation is not just quota math. A single IP submitting repeated 50-item POSTs to a bulk endpoint looks like a script in a way that slow, browsing-like GET traffic does not. Some targets flag bulk-endpoint abuse faster than page-view patterns. Batching can be stealthier on the counter and more suspicious on the pattern. There is no universal answer; you have to know the target.

When a target does rate-limit you, read the headers:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 3
X-RateLimit-Reset: 42
Retry-After: 12

Interpretation: 100 requests allowed per window, you have 3 left, the window resets in 42 seconds, and the server is explicitly asking you to wait 12 seconds before the next request. Honoring Retry-After is not optional politeness — ignoring it is the fastest way to convert a soft limit into a hard block.

A concurrency limiter tuned for each pattern, using httpx and anyio:

import asyncio
import httpx
import anyio

async def sequential_limited(ids, rpm=60):
    """One request per interval, spread evenly."""
    interval = 60 / rpm
    async with httpx.AsyncClient() as client:
        for pid in ids:
            r = await client.get(
                f"https://store.example.com/products/{pid}", timeout=15
            )
            await asyncio.sleep(interval)
            yield pid, r

async def batched_limited(ids, batch_size=50, batches_per_min=2):
    """One batch per interval -- 100 items/min at 50 per batch."""
    interval = 60 / batches_per_min
    async with httpx.AsyncClient() as client:
        for i in range(0, len(ids), batch_size):
            r = await client.post(
                "https://store.example.com/api/products/batch",
                json={"ids": ids[i:i + batch_size]},
                timeout=60,
            )
            await asyncio.sleep(interval)
            yield ids[i:i + batch_size], r

Pattern	Requests/min at 100 items/min	Quota consumption	Detection profile
Sequential	100	100% of a 100/min quota	Looks like heavy browsing; easy to throttle via headers
Batched (50/call)	2	2% of the same quota	Looks like API automation; bulk endpoints sometimes monitored separately

If you are routing through a scraping API rather than hitting the target directly, the same calculus applies to your token spend — batch submissions through a service-side queue (like the batch endpoint on POST /api/v1/async/batch) keep your client-side request count trivially low while the service manages per-target pacing. For very large jobs, the async batch pattern described in async scraping at scale handles the queueing, webhooks, and per-job status for you.

Decision Framework: Matching the Pattern to Job Type, Data Freshness, and Budget

Everything above collapses into a decision table. Mine, after running both patterns in production pipelines:

Job archetype	Recommended pattern	Batch size	Retry policy
Nightly catalog sync	Batched	25–50	Per-item retry with backoff; dead-letter not_found IDs
Real-time price/stock check	Single calls	1	Immediate retry, max 2; fail loudly to caller
One-off historical backfill	Batched	50–100	Per-item retry, aggressive; log everything
Freshness-critical alerts (price drops)	Single calls	1	Retry once, then escalate to next data source
New target, unknown error profile	Single calls first	1	Collect failure stats for a week, then decide

The best real-world setups are hybrids. A catalog pipeline for store.example.com might run a nightly batched sync across all category pages — 50 IDs per call to the bulk endpoint, per-item retries, dead-ID store — while a parallel single-call loop refreshes the 200 hottest SKUs every 15 minutes for stock accuracy. The batched path optimizes cost per item; the single-call path optimizes freshness and precise failure attribution. Neither pattern alone gives you both.

Before committing to a pattern on a new domain, answer four questions:

1. What is the per-item failure rate? Above roughly 10%, all-or-nothing batching stops being viable. Measure it with sequential calls first.
2. Does the target’s bulk endpoint bill (or fail) per item or per request? This single fact determines your cost ceiling and your retry design.
3. How fresh does the data need to be? If minutes matter, single calls with tight timeouts beat fat batches whose queue time you cannot control.
4. What happens downstream when an item is missing? If a missing item triggers a wrong decision (pricing, purchasing), you need per-item failure attribution — which means single calls or per-item batch semantics.

An opinion to close the decision section, and one you might disagree with: default to single calls. Batching is an optimization you earn after you understand a target’s failure profile, not a starting point. I have watched teams ship batched pipelines on day one, hit the malformed-ID-poisons-the-batch problem in week two, and spend a month building bisection retry logic that a sequential loop never needed. Start sequential, measure, then batch the parts of the job where the numbers justify the complexity.

Wrap-up

Batching and single calls are not competing philosophies — they are tools with different failure semantics. Batching wins on request count, quota consumption, and amortized overhead, with realistic speedups in the 5x–15x range rather than the 50x people assume. Single calls win on failure isolation, retry simplicity, freshness, and predictable cost under success-based billing.

The billing model is the twist most teams miss. When you pay only for successful requests, failed single calls are free tuition about the target’s error profile, while a poisoned batch can waste real work. Run the simulation, measure your per-item failure rate, and let the numbers pick the pattern — then build the hybrid that uses each where it is actually strong.