CSS Selectors vs LLM Extraction: What Breaks First

Selectors vs LLM Extraction: Which Breaks First?

CSS selectors fail loudly. LLM extraction fails quietly. That one difference explains most of the practical decisions you’ll make about extraction architecture, and the rest of this comparison is really just working through the consequences of it. Here’s the short version up front:

Dimension	CSS/XPath selectors	LLM extraction
Marginal cost per page	Effectively $0	Fractions of a cent, billed per run
Latency	Milliseconds	Seconds
Breaks when markup changes	Immediately, visibly	Slowly, invisibly
Breaks when wording changes	Mostly unaffected	Occasionally confused
Wrong-value risk	Low (fixed nodes)	Real (plausible output, wrong content)
Typical fix	Rewrite a selector	Iterate a prompt or schema

Now the long version, with code.

Failure Modes: Hard Crashes vs Quiet Hallucinations

Run a selector against markup it doesn’t recognize and you get None. That None then travels downstream until something touches it:

card = extract_with_selectors(html)          # selector for .price-now missed
price = float(card["price"].replace("$", ""))
# TypeError: 'NoneType' object has no attribute 'replace'

Ugly, but honest. The stack trace points at the exact line, the exact field, the exact moment. Your pipeline turns red and someone gets paged.

The LLM path on the same broken page produces something much worse:

{
  "title": "Aeroline Wireless Mouse M420",
  "price": "$39.99",
  "list_price": "$29.99",
  "rating": "4.6",
  "availability": "in stock"
}

Well-formed. Correctly typed. Valid against your schema. Also wrong — the model grabbed the compare-at price as the current price and swapped the two fields. No exception fires. Every dashboard stays green. Three weeks later someone asks why the average discount on store.example.com went negative.

Mapping failure types to extractors:

Failure type	Who produces it
Empty result (null fields)	Selectors — their signature move
Exception / crash	Selectors (via null propagation); rare for LLMs
Wrong value in a plausible shape	LLMs, overwhelmingly
Partial value (truncated, missing units)	Both, roughly equally
Fields swapped or mislabeled	LLMs only

Here’s the uncomfortable part: the loud failure is the cheap one. A crash costs you an hour. A hallucination costs you the trust anyone placed in the dataset, because you usually find out from a stakeholder, not from a monitor.

Accuracy Baseline: One Product Page, Two Extractors

Before arguing about drift, establish what each method does on clean, frozen markup. Take this product card:

<article class="product-card" data-sku="M420">
  <h1 class="product-title">Aeroline Wireless Mouse M420</h1>
  <span class="price-now">$29.99</span>
  <span class="price-was">$39.99</span>
  <span class="rating">4.6</span>
  <span class="availability">In stock</span>
</article>

The selector implementation:

from bs4 import BeautifulSoup

SELECTORS = {
    "title":        "h1.product-title",
    "price":        "span.price-now",
    "list_price":   "span.price-was",
    "rating":       "span.rating",
    "availability": "span.availability",
}

def extract_with_selectors(html: str) -> dict:
    soup = BeautifulSoup(html, "html.parser")
    return {
        name: (node.get_text(strip=True) if (node := soup.select_one(sel)) else None)
        for name, sel in SELECTORS.items()
    }

The LLM equivalent — a JSON Schema sent alongside the fetch instead of selectors:

import requests

resp = requests.post(
    "https://api.finedata.ai/api/v1/scrape",
    headers={"Authorization": "Bearer fd_your_api_key"},
    json={
        "url": "https://example.com",
        "only_main_content": True,   # trims nav/footer, cuts tokens sent to the model
        "extract_schema": {
            "type": "object",
            "properties": {
                "title":        {"type": "string"},
                "price":        {"type": "number"},
                "list_price":   {"type": "number"},
                "rating":       {"type": "number"},
                "availability": {"type": "string",
                                 "enum": ["in stock", "out of stock", "preorder"]}
            },
            "required": ["title", "price", "rating"]
        },
    },
)
record = resp.json()["data"]["extract"]

One nice property of this setup: the same endpoint accepts extract_rules (selectors, executed service-side) or extract_schema (model-driven extraction), so A/B-ing the two approaches against identical pages is trivial with FineData or any provider that offers both. Expected output from the schema run:

{"title": "Aeroline Wireless Mouse M420", "price": 29.99,
 "list_price": 39.99, "rating": 4.6, "availability": "in stock"}

Illustrative run from the harness in the last section, over 100 saved snapshots (one site, one run — treat as a template, not a benchmark):

Metric	Selectors	LLM (extract_schema)
Field-level accuracy	400/400	396/400
Null fields	0	2 (a missed rating, twice)
Incorrect values	0	2 (one availability phrasing oddity)
Extraction latency	~4 ms/page	~1.9 s/page

The 400/400 for selectors is almost circular — the selectors were written against these exact snapshots. That’s the point: on frozen markup, selectors are perfect and free. Everything interesting happens when the markup moves. More on schema-driven extraction generally lives in From HTML to JSON: Schema-Driven Extraction.

Drift Test: What Breaks When store.example.com Redesigns

The redesign lands. Compare the card:

<!-- v1 -->
<span class="price-now">$29.99</span>
<span class="price-was">$39.99</span>

<!-- v2: price moves into a data attribute, classes renamed -->
<article class="product-card" data-sku="M420"
         data-price="29.99" data-list-price="39.99">
  <h1 class="pd-title">Aeroline Wireless Mouse M420</h1>
  <div class="pd-pricing">
    <span class="pd-current">$29.99</span>
    <s class="pd-msrp">$39.99</s>
  </div>

Selector output on v2:

>>> extract_with_selectors(html_v2)
{'title': None, 'price': None, 'list_price': None,
 'rating': None, 'availability': None}

Total loss, instantly detectable. The LLM on v2 still returns a price — most of the time the right one. But when the visible labels are gone or renamed, it occasionally reads pd-msrp as the current price and ships $39.99 with full confidence. Degraded, not dead, and far harder to notice.

Redesign scenario	Selectors	LLM extraction
Class rename (.price-now → .pd-current)	Fail (null)	Pass
Restructure (price into data-price attribute)	Fail (null)	Pass, occasionally grabs compare-at
Promo banner injected above card (“$5 off today”)	Pass (anchored to card)	Degraded — banner price leaks into output
Full template swap (new markup, new wording)	Fail	Degraded — field semantics drift

Note row three. It’s the case nobody advertises: when the page gets noisier rather than restructured, the selector’s rigidity becomes an advantage. It cannot be distracted by a marketing banner. The LLM can.

The Real Cost Model: Cents per 1,000 Pages vs Free but Fragile

Selectors cost nothing to run at extraction time. LLM extraction bills on every invocation. Assume a mid-tier model, ~2,000 input + 300 output tokens per page after only_main_content trimming, at $0.15/M input and $0.60/M output — that’s $0.00048 per page, or $0.48 per 1,000 pages. Swap in your own model’s pricing; the shape of the math won’t change.

Volume	Tokens/page	Price per 1k pages	Monthly bill (30 days)
1k pages/day	~2,300	$0.48	~$14
10k pages/day	~2,300	$0.48	~$144
100k pages/day	~2,300	$0.48	~$1,440

Volume	Selector extraction cost
Any of the above	~$0.00 — lxml parses a 120 KB page in a few ms; one modest box handles millions/month

Fetch and proxy spend is identical for both methods, so it cancels out of the comparison.

Now the crossover calculation. Price engineering time at $60/hour. One hour buys $60 / $0.48-per-1k ≈ 125,000 pages of LLM extraction. Two conclusions fall out:

• Below roughly 4,000 pages/day, a full month of pure-LLM extraction costs less than a single hour of your time. If you’d burn more than an hour a month maintaining selectors, LLM everywhere is defensible.
• At 100k pages/day, the ~$1,440 monthly bill buys about 24 hours of engineering. If selector upkeep costs you less than that per month — and for most stable sites it does — selectors win on money alone.

That asymmetry surprises people. The per-run billing that looks trivial at prototype scale compounds into a salary at production scale.

Maintenance Math: Time-to-Detect and Time-to-Fix per Incident

Selector breaks announce themselves, but only if you’re listening on the right channel. Here’s a real-shaped log sequence from a pipeline that alerted on HTTP status but not on field nulls:

[02:14:03] store=store.example.com pages=312 http=200 null_rate[price]=0.51
[02:29:41] store=store.example.com pages=305 http=200 null_rate[price]=0.53
[03:02:19] store=store.example.com pages=298 http=200 null_rate[price]=0.49
[04:40:55] store=store.example.com pages=301 http=200 null_rate[price]=0.50
[06:15:52] ALERT store=store.example.com null_rate[price]=0.52 threshold=0.05 window=1h

The redesign shipped at 02:14. The alert fired at 06:15 — four hours late, because nothing watched extraction fields. The fix is ten lines:

def page_oncall(message: str) -> None:
    """Route an alert to the on-call channel. Wire this to PagerDuty, Slack, or wherever your team gets paged."""
    print(f"ALERT: {message}")

FIELD_THRESHOLDS = {
    "store.example.com": {"price": 0.05, "title": 0.02, "rating": 0.10},
}

def check_batch(store: str, records: list[dict]):
    if not records:
        return  # empty batch: nothing scraped, nothing to alert on
    for field, limit in FIELD_THRESHOLDS[store].items():
        nulls = sum(1 for r in records if r.get(field) is None)
        if nulls / len(records) > limit:
            page_oncall(f"{store}: null_rate[{field}]={nulls/len(records):.2f}")

Incident	Selector break	LLM drift
Detection	Minutes, via null-rate alert	Days to weeks, via sampling audit
Diagnosis	10–30 min: diff markup, find new node	Hours: review outputs, guess which cue misled the model
Fix	1–3 hours incl. regression test	Prompt/schema iteration; non-deterministic, may regress elsewhere
Blast radius	Affected fields vanish loudly	A few percent of values are wrong silently

The selector incident is annoying but bounded. The LLM incident is open-ended because “the model was sometimes confused” has no one-line fix. Pairing extraction with change detection on the raw HTML shortens both timelines.

Silent Failures: When Extraction Succeeds but the Data Is Wrong

The worst case isn’t a crash. It’s this:

// LLM output
{"price": 39.99, "list_price": 29.99, "discount_label": "Save $10"}

// Ground truth
{"price": 29.99, "list_price": 39.99, "discount_label": "Save $10"}

Sale and list price, swapped. Both numbers were on the page; the model assigned them backwards; every type check passes. Downstream, “discount” computes as negative 33%, and if nothing validates that, it lands in the warehouse.

The guard that catches it is almost embarrassingly simple — but it has to fail loudly on every malformed shape the model can emit, including nulls, or the quiet failure slips through the guard too:

def validate(rec: dict) -> bool:
    price = rec.get("price")
    if price is None or not isinstance(price, (int, float)) or price <= 0:
        return False

    rating = rec.get("rating")   # explicit None check: the model emits nulls
    if rating is None or not isinstance(rating, (int, float)) or not 0 <= rating <= 5:
        return False

    list_price = rec.get("list_price")
    if list_price is not None:
        if not isinstance(list_price, (int, float)) or price > list_price:
            return False
        discount = 1 - price / list_price
        if not 0 <= discount < 0.80:
            return False
    return True

Guardrail	Catches	Cost
Schema validation (types, required, enums)	Missing or ill-typed fields	Near zero
Range checks (price > 0, rating ≤ 5)	Swapped or absurd values	Near zero
Cross-field rules (sale ≤ list, discount < 80%)	The swap above	Near zero
Sampling audits (N random pages/week, human diff)	Semantically wrong extractions	Minutes per week
HTML hash change detection	Upstream redesign, before it bites	Cheap

Run the first three on every record regardless of extractor. Selectors need them less, but “less” is not “never” — a renamed class can make .price-was match a shipping cost.

A Hybrid Router: Selectors First, LLM Fallback on Low Confidence

The production pattern I’d actually run: selectors as the default path, LLM as the exception handler.

import requests

REQUIRED = ["title", "price", "list_price", "rating", "availability"]

def extract(url: str, html: str) -> tuple[dict, str]:
    rec = extract_with_selectors(html)
    missing = [f for f in REQUIRED if rec.get(f) is None]

    if len(missing) / len(REQUIRED) <= 0.25 and validate(rec):
        return rec, "selectors"

    # fallback: re-scrape with schema extraction
    llm_rec = requests.post(
        "https://api.finedata.ai/api/v1/scrape",
        headers={"Authorization": "Bearer fd_your_api_key"},
        json={"url": url, "only_main_content": True, "extract_schema": SCHEMA},
    ).json()["data"]["extract"]
    if validate(llm_rec):
        return llm_rec, "llm"
    quarantine(url, llm_rec)   # never ship unvalidated fallbacks
    raise ValueError(f"unvalidated fallback record for {url}")

At a 5% fallback rate the blended extraction price is 0.05 × $0.48 = $0.024 per 1,000 pages — about $7/month at 10k pages/day, versus ~$144 pure-LLM and $0-plus-breakage pure-selector.

Strategy	$/1k pages	Monthly @ 10k/day
Pure selectors	$0.00	$0 + breakage risk
Hybrid, 5% fallback	$0.024	~$7
Pure LLM	$0.48	~$144

How to route field types:

Field type	Method
Stable, well-anchored (title, SKU, price on known markup)	Selectors
Volatile but regular (promo badges, availability wording)	Selectors + LLM fallback
Unstructured prose (descriptions, review text)	LLM, no contest
A site you’ll scrape fewer than a few dozen times	LLM — you’ll never recoup selector development

And here’s where you may disagree with me: for stable fields at volume, I think pure-LLM extraction is a regression, not an upgrade. You trade a loud, cheap, fixable failure for a quiet one and pay rent on every page forever. The model’s tolerance for markup drift is genuinely valuable — but as insurance, not as the primary engine.

Build Your Own Break-Point Harness

Don’t take my tables on faith. Reproduce them on your own targets in an afternoon:

# harness.py — both extractors vs labeled ground truth over saved snapshots
import json
from pathlib import Path

import requests

SNAPSHOTS = Path("snapshots/store.example.com")
LABELS = json.loads((SNAPSHOTS / "labels.json").read_text())  # \{file: \{url, fields}}

def run_selectors(html: str) -> dict: ...

def run_llm(url: str) -> dict:
    return requests.post(
        "https://api.finedata.ai/api/v1/scrape",
        headers=\{"Authorization": "Bearer fd_your_api_key"},
        json=\{"url": url, "only_main_content": True, "extract_schema": SCHEMA},