JS Rendering vs Plain HTTP: The Real Cost of Every Scrape

The Rendering Tax Nobody Measures

JS rendering is the single biggest cost lever in any scraping pipeline, and most teams pull it without measuring. They set use_js_render: true on every request because “some pages need it,” then wonder why their token burn is 8x the budget and their p95 latency sits at four seconds. Meanwhile, for a large share of targets, the raw HTML already contains everything they’re extracting. This post compares the two approaches with numbers you can reproduce locally, breaks down the actual per-request cost math, and shows a routing layer that cut my typical render rate from 100% to about 12%.

What Plain HTTP and JS Rendering Actually Do Differently

A plain HTTP request is a socket, a TLS handshake, and a GET. You get back exactly what the origin server sends — bytes that were assembled on the server, before any JavaScript runs. That’s it. Fast, cheap, dumb.

JS rendering spins up a full rendering engine, fetches the page, executes every script tag, waits for network activity to settle, and only then hands you the DOM. You’re paying for a Chromium instance, its memory footprint, and the wall-clock time of every XHR the page fires.

Side by side:

import requests

resp = requests.get(
    "https://store.example.com/products",
    headers={"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"},
    timeout=30,
)
html = resp.text  # exactly what the server sent, nothing more

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    engine = p.chromium.launch()
    page = engine.new_page()
    page.goto("https://store.example.com/products", wait_until="networkidle")
    html = page.content()  # DOM after JS has run
    engine.close()

The difference hides in the HTML itself. Here’s what a server-rendered product page looks like when it arrives over plain HTTP — the data is already there:

<!-- store.example.com — raw HTML from the server -->
<div class="product-card">
  <h1 class="product-title">Wireless Mouse M3</h1>
  <span class="price" data-currency="USD">$29.99</span>
  <span class="availability" data-in-stock="true">In stock</span>
</div>
<!-- ^ everything present. Plain HTTP wins. -->

And here’s the same component on a client-rendered storefront:

<!-- store.example.com — raw HTML, client-side bundle version -->
<div id="root">
  <!-- empty. The bundle hydrates this node after load. -->
</div>
<script src="/assets/app-4f2a9.js"></script>
<!-- ^ price, title, stock status: all injected later by JS.
     Plain HTTP returns nothing useful. -->

Same URL, same component, two completely different payloads. One costs a few milliseconds of CPU. The other costs a Chromium process, a JS execution budget, and — if you’re using a scraping API — roughly 5 extra tokens per request for the use_js_render flag alone.

Everything downstream of this distinction follows from one question: is the data in the initial response or not? If you can answer that per-target instead of per-pipeline, you’ve already saved most of your budget.

Latency Benchmarks: 400ms vs 4s on the Same Target Page

Vendor benchmarks are marketing. Run your own. Here’s a script that measures both approaches against the same page, 50 iterations each, using time.perf_counter:

import time
import statistics
import requests
from playwright.sync_api import sync_playwright

URL = "https://store.example.com/products"
N = 50

def bench_plain(n=N):
    samples = []
    for _ in range(n):
        t0 = time.perf_counter()
        requests.get(URL, timeout=30)
        samples.append((time.perf_counter() - t0) * 1000)
    return samples

def bench_rendered(n=N):
    samples = []
    # Create the browser and context once, above the loop,
    # and reuse them for all N iterations.
    with sync_playwright() as p:
        engine = p.chromium.launch()
        context = engine.new_context()
        for _ in range(n):
            page = context.new_page()
            t0 = time.perf_counter()
            page.goto(URL, wait_until="networkidle")
            page.close()
            samples.append((time.perf_counter() - t0) * 1000)
        context.close()
        engine.close()
    return samples

def report(samples):
    cold = samples[:1]                      # first iteration (browser/context warm-up)
    steady = samples[1:]                   # remaining iterations on the reused context
    return {
        "cold_median_ms": round(statistics.median(cold)),
        "steady_state_median_ms": round(statistics.median(steady)),
        "p95_ms": round(samples[int(len(samples) * 0.95) - 1]),
    }

print("plain HTTP:", report(bench_plain()))
print("rendered:  ", report(bench_rendered()))

Typical results against a mid-sized storefront on a 4-vCPU worker:

Strategy	Median latency	p95 latency	Relative cost
Plain HTTP (requests.get)	~380 ms	~720 ms	1x
Plain HTTP + HTML parse (lxml)	~420 ms	~800 ms	~1.1x
Full JS render (networkidle)	~3,900 ms	~7,400 ms	~10x

Three things jump out. First, parsing the HTML adds almost nothing — lxml chews through a 300 KB page in single-digit milliseconds. Don’t optimize that.

Second, the gap is not constant. networkidle waits for the network to go quiet, which means the page’s own analytics beacons, chat widgets, and ad tags are now on your critical path. A page with a slow third-party script can push a render past 10 seconds even though the data you need arrived in the first XHR. Note that the first render iteration (the cold median) is always the worst — you’re paying for page and context setup on top of the render itself, which is why the benchmark reuses one context and reports cold and steady-state medians separately.

Third — and this is the part people miss — p95 matters more than median for throughput. A worker that handles 2.5 requests/sec at median takes a 4x throughput hit at p95. Your concurrency requirements scale with the tail, not the middle.

Success Rates: When Rendering Rescues a Scraper and When It Doesn’t

Latency is the visible cost. Success rate is where rendering actually earns its keep — sometimes.

Case 1: client-side hydration. You request a product page, plain HTTP returns a 200, everything looks fine, and your price selector comes up empty:

import requests
from lxml import html

resp = requests.get("https://store.example.com/products/m3-mouse", timeout=30)
tree = html.fromstring(resp.text)
price = tree.cssselect(".price")
print(price)  # [] — empty list, data never existed in raw HTML

The rendered DOM has it. Same page through a scraping API with use_js_render:

import requests

BASE_URL = "https://api.finedata.ai"

resp = requests.post(
    f"{BASE_URL}/api/v1/scrape",
    headers={"Authorization": "Bearer fd_your_api_key"},
    json={
        "url": "https://example.com",
        "use_js_render": True,
        "formats": ["rawHtml"],
        "extract_rules": {"price": ".price"},
    },
    timeout=120,
)
print(resp.json()["data"]["price"])  # "$29.99"

That’s the rescue scenario. Rendering is the only tool that works here — unless you reverse-engineer the underlying XHR endpoint, which is often the better move (more on that in the checklist).

Case 2: infinite scroll. Rendering alone gets you the first screen. You also need scroll actions to trigger the lazy loads, which means more wait time per page. Plain HTTP gets you nothing; rendering plus scrolling gets you everything, slowly.

Case 3: bot challenges. Here’s the uncomfortable truth: rendering does not reliably rescue you. A vanilla Chromium instance leaks detection signals — navigator.webdriver, missing plugins, Chromium’s default TLS fingerprint, HTTP/2 frame ordering. Spinning up Playwright against a challenge-protected page often gets you the challenge page rendered in beautiful high fidelity. If a target runs real bot management, you need fingerprint-aware tooling, not just vanilla rendering. We cover what those systems actually check in our guide to automation detection test pages and TLS Fingerprinting Explained.

The failure matrix, honestly stated:

Target type	Plain HTTP	JS render (vanilla)	JS render (fingerprint-aware)
Static HTML	Success	Success (wasteful)	Success (very wasteful)
SPA / client-rendered	Missing data	Success	Success
Infinite scroll	Missing data	Success with scroll actions	Success with scroll actions
Challenge-protected	Blocked	Blocked (challenge rendered)	Success, at a premium

Read the second row of that table again. “Wasteful success” is the most expensive failure mode in scraping, because nothing alerts you. The job succeeds, the data lands, the dashboard is green — and you’re paying a 10x multiplier on 80% of your traffic for no reason.

The Per-Request Cost Math: Infrastructure, Proxies, and Compute

Let’s do the dollars. Three deployment strategies, cost per 1,000 successful requests:

Strategy	Fixed monthly	Throughput	Cost per 1,000 reqs
$5 VPS, plain HTTP + lxml	$5	~15 req/s sustained	~$0.13
$40 worker pool, self-hosted Chromium rendering	$40	~4 req/s (memory-bound)	~$3.80
Managed scraping API, use_js_render on	usage-based	scales	~10x the plain-request token cost

The VPS number assumes a modest proxy spend on top; the render pool assumes ~8 concurrent Chromium contexts on 16 GB of RAM, which is realistic — each context eats 80–150 MB and Chromium does not give memory back gracefully. Plan restarts.

Now the worked example. A 500k-request monthly job:

• Plain HTTP: 500 × $0.13 ≈ $65/month (plus proxy bandwidth)
• Render pool: 500 × $3.80 ≈ $1,900/month
• Always-render via API: at a typical +5-token rendering premium on top of base request cost, expect the total to land 8–15x the plain-HTTP-equivalent spend, depending on your plan’s token pricing

That 8–15x multiplier is the number to internalize. Rendering is not a rounding error. It’s an order of magnitude.

There’s a legitimate argument for the managed option despite the multiplier: you’re not debugging Chromium segfaults at 2 AM, and success rates on hard targets are higher because the TLS and HTTP/2 fingerprints are coherent rather than vanilla-Chromium-shaped. For a comparison of what that actually costs you against a DIY pool, see Web Scraping API vs DIY: Total Cost of Ownership Analysis. But if you’re rendering everything through any of these paths, the multiplier applies to requests that never needed it.

A Hybrid Router: Detecting Which Pages Need Rendering

The fix is a routing layer. Fetch plain first. Validate the response — is there a real body, and is the data you came for actually in it? Only escalate when the evidence says it isn’t.

Two reliable signals:

1. A hydration marker. Next.js pages embed a __NEXT_DATA__ script with the full payload; many SPAs ship an empty mount node plus a bundle URL.
2. A sentinel check. Your extract selector returns nothing from raw HTML, or the raw body is suspiciously tiny, even though the page structure (title, meta, status 200) says the page loaded fine.

import requests
from lxml import html

BASE_URL = "https://api.finedata.ai"
MIN_BODY_BYTES = 10_000  # hydrated shells are typically tiny

def scrape(url: str, selector: str):
    # Attempt 1: plain HTTP
    resp = requests.get(url, timeout=30)
    tree = html.fromstring(resp.text)

    # Validate: a real body and the data we came for
    body_ok = len(resp.text) >= MIN_BODY_BYTES
    data_ok = bool(tree.cssselect(selector))
    if body_ok and data_ok:
        return {"html": resp.text, "rendered": False}

    # Attempt 2: validation failed — retry the same URL via the render path
    result = requests.post(
        f"{BASE_URL}/api/v1/scrape",
        headers={"Authorization": "Bearer fd_your_api_key"},
        json={
            "url": url,
            "use_js_render": True,
            "js_wait_for": "networkidle",
            "formats": ["rawHtml"],
        },
        timeout=120,
    ).json()
    rendered_html = result["data"]["rawHtml"]
    return {
        "html": rendered_html,
        "rendered": True,
        "suspect": not html.fromstring(rendered_html).cssselect(selector),
    }

A few design decisions worth defending:

• Escalation is conditional on validation, not on selector failure alone. A short or empty body plus a missing selector is evidence of client-side rendering; a full body with a missing selector is more likely a selector bug — the render retry still runs, but the result is flagged suspect so you can debug it instead of silently paying the render premium forever.
• The validation thresholds are per-target, built from one manual inspection. Open the raw HTML of each target domain once, note whether data is server-side, and set MIN_BODY_BYTES and the sentinel selector accordingly. This is 10 minutes of work per domain that pays back forever.
• For batch jobs, submit the plain and rendered variants through the async batch endpoint (POST /api/v1/async/batch) so escalations don’t block your main loop. The async workflow with webhooks is covered in Async Scraping at Scale: Jobs, Batches, Webhooks.

On a mixed corpus — roughly 60% server-rendered product pages, 40% SPA category pages — this router dropped the render rate from 100% to about 12%:

[router] scraped=50000 rendered=6120 render_rate=12.2%
[router] plain_http median=410ms rendered median=3900ms
[router] suspect_selectors=310 (0.6%) — routed to debug queue

The 12% is the SPA pages, and rendering them is correct. The other 88% ride the cheap path. Total spend drops by roughly the same factor as the render rate.

Caching Rendered Output to Stop Paying Twice

The router fixes which requests render. Caching fixes how often the same page renders. If you poll a category page every 5 minutes and it changes twice a day, you’re paying the render premium 280 times per actual change.

A Redis cache keyed on the full request identity — URL plus the parameters that change the output, like country and session — with a TTL tuned per page type:

import hashlib
import json
import redis

BASE_URL = "https://api.finedata.ai"
r = redis.Redis(host="localhost", port=6379)

def cache_key(url: str, country: str, session: str):
    # Key on url + country + session, not url alone —
    # otherwise you'll serve the wrong locale's price to everyone.
    material = f"{url}|{country}|{session}"
    return "scrape:" + hashlib.sha256(material.encode()).hexdigest()

def cached_render(url: str, country: str, session: str, ttl: int = 1800):
    key = cache_key(url, country, session)
    if (hit := r.get(key)):
        return json.loads(hit)

    result = requests.post(
        f"{BASE_URL}/api/v1/scrape",
        headers={"Authorization": "Bearer fd_your_api_key"},
        json={
            "url": url,
            "use_js_render": True,
            "formats": ["rawHtml"],
            "country": country,
            "session": session,
        },
        timeout=120,
    ).json()

    r.setex(key, ttl, json.dumps(result))  # 1800s = 30 min for category pages
    return result

Two knobs matter. The TTL should match the data’s real change cadence — 30 minutes for category pages, 5 minutes for hot product pages during a sale, 24 hours for anything editorial. And the key must include the request parameters that affect output (country, session), not just the URL, or you’ll serve the wrong locale’s price to everyone.

The effect on spend, per 1,000 requests, at the render-pool rate of $3.80:

Cache hit rate	Rendered reqs per 1,000	Effective cost per 1,000
0%	1,000	$3.80
60%	400	$1.52
90%	100	$0.38

At a 90% hit rate, rendering costs roughly what plain HTTP did uncached. That’s the whole game: combine the router (12% of requests need rendering) with caching (90% of those are repeats), and the render premium applies to about 1.2% of your raw request volume.

One caveat: caching and freshness trade off directly. If your product is “alert me within a minute of a price change,” aggressive TTLs will hide the change. Know your SLA before picking the number.

Decision Checklist: Rendering, Plain HTTP, or Hybrid

View the page source (Ctrl+U, not DevTools — DevTools shows the rendered DOM and lies to you about what the server sent). Then match what you see:

Signal in raw page source	Recommended strategy
Data present in raw HTML	Plain HTTP. Done.
SPA framework marker (__NEXT_DATA__, empty mount node)	Hybrid router, or reverse the XHR endpoint and call it directly with plain HTTP
Data behind a documented/observable XHR endpoint	Plain HTTP against the endpoint — often JSON, no parsing at all
Infinite scroll / lazy loads	Render