Measure Exit IP Geolocation Accuracy With a Local Check

What “Exit IP Geolocation Accuracy” Means and Why Local Logging Settles It

You asked for a German exit IP. The storefront served French pricing, availability from the wrong warehouse, and a locale your pipeline happily parsed as if it were the page you expected. Nothing errored. Status was 200, your selectors matched, and the off-target data flowed straight into your dataset. That is the failure mode this audit catches — and it is invisible unless you measure it.

Define the metric narrowly: exit IP geolocation accuracy is the fraction of requests where the country your exit IP resolves to equals the country you asked for, measured at request time, from your own log. Not what the proxy dashboard claims. Not what the routing layer intends. The comparison happens on two labels attached to every single request:

+------------+     +-------------+     +----------------+     +----------------+
| audit.py   | --> | proxy pool  | --> | exit IP        | --> | example.com    |
| wants "DE" |     | country     |     | 203.0.113.7    |     | sees the exit  |
|            |     | routing     |     | resolves: "FR" |     | IP, not you    |
+------------+     +-------------+     +----------------+     +----------------+
       |                    |                    |
       |              label A: requested    label B: resolved
       |                country = "DE"        country = "FR"
       |
       +---------- compare A vs B, per request, in your log ----------+

Here is what those two labels look like against a third one — the country the provider claims you got:

exit IP	requested	provider-claimed	geo-lookup at request time	verdict
198.51.100.23	DE	DE	DE	match
203.0.113.7	DE	DE	FR	miss — claim and reality diverged
192.0.2.14	US	US	(no entry)	unresolved — excluded from the score

A provider dashboard will happily report the middle column. It averages over windows you don’t control, hides per-session rot, and lags routing changes by days. Geolocation databases also disagree with each other — an IP can be DE in one database and FR in another the same afternoon. The only comparison that pays rent is the one performed at request time, on your requests, against a database you chose and whose freshness you control. Everything below builds exactly that.

Setting Up the Audit Script Skeleton with Requests and a Per-Run Log File

The skeleton is deliberately boring: one function that fetches, one that resolves, one that appends a JSON Lines row. JSONL because it is append-only, greppable, and streams into pandas later without a parser.

import json
import re
import time
from datetime import datetime, timezone

import requests

REQUESTED_COUNTRY = "DE"

# Most geo-routing proxy gates encode the country in the username.
# Replace host and credentials with your own.
PROXY_CONFIG = {
    "http": f"http://audit-user-country-{REQUESTED_COUNTRY.lower()}:REPLACE_ME@gate.provider.example:8000",
    "https": f"http://audit-user-country-{REQUESTED_COUNTRY.lower()}:REPLACE_ME@gate.provider.example:8000",
}

LOG_PATH = "geo_audit.jsonl"
IP_RE = re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b")

def first_ipv4(text):
    if not text:
        return None
    m = IP_RE.search(text)
    return m.group(0) if m else None

def fetch_once(url, timeout=10):
    """One scrape through the proxy chain.
    Returns (response, timestamp, latency_ms)."""
    started = time.monotonic()
    resp = requests.get(url, proxies=PROXY_CONFIG, timeout=timeout)
    ts = datetime.now(timezone.utc)
    latency_ms = round((time.monotonic() - started) * 1000)
    return resp, ts, latency_ms

def append_log(row, path=LOG_PATH):
    with open(path, "a") as fh:
        fh.write(json.dumps(row) + "\n")

Note that fetch_once threads its url argument all the way into requests.get — the URL you pass is the URL that is fetched and logged, which is what makes the target field trustworthy. Each request produces one line shaped like this (timestamp trimmed for display — the real file carries full ISO timestamps):

{"ts":"09:14:02Z","target":"https://example.com","requested_country":"DE","exit_ip":"198.51.100.23","resolved_country":"DE","matched":1,"latency_ms":640,"session_id":"geo-de-01","status_code":200}

Two decisions worth defending. First, matched is 1, 0, or null — never a boolean — because unresolved lookups need their own lane, which we get to below. Second, the echo target should ideally be an endpoint you control, because the audit itself tells a third party when and how often you probe. If you have never logged exit IPs at all, start with the simpler exercise in Log Your Exit IPs: Measure Proxy Rotation Locally; this post is the geo layer on top of that habit.

Resolving the Exit IP’s Country on Every Scrape Request

To score a request you need the exit IP’s country at that moment. I use a local MaxMind GeoLite2 country database via geoip2 rather than a live lookup API, and I will argue for that choice: a local file adds no per-request third-party dependency, costs nothing per lookup, runs offline, and doesn’t leak your audit cadence to anyone. The trade-off is freshness — the database can lag reality by up to a week, so cross-check a sample of mismatched IPs against a second source before blaming your pool.

import geoip2.database

GEOIP_DB_PATH = "/usr/local/share/GeoIP/GeoLite2-Country.mmdb"
_reader = geoip2.database.Reader(GEOIP_DB_PATH)

def resolve_country(ip):
    if not ip:
        return None
    try:
        return _reader.country(ip).country.iso_code
    except geoip2.errors.AddressNotFoundError:
        return None

Capturing the exit IP itself has a primary path and a fallback. Some origins echo your client IP in a response header; most don’t, so the fallback hits an IP echo endpoint through the same proxy chain:

def capture_exit_ip(primary_response, proxies, timeout=10):
    # 1) Did the target tell us? Some origins echo the client in a header.
    for header in ("x-client-ip", "x-served-by", "x-originating-ip"):
        ip = first_ipv4(primary_response.headers.get(header, ""))
        if ip:
            return ip
    # 2) Fallback: ask an echo endpoint through the same proxy chain.
    r = requests.get("https://api.example.com/ip", proxies=proxies, timeout=timeout)
    return first_ipv4(r.text)

Keep the database honest with a weekly refresh:

# /etc/cron.d/geoip-update -- refresh the country DB weekly
17 4 * * 1 /usr/local/bin/geoipupdate --config /etc/geoipupdate.conf >> /var/log/geoipupdate.log 2>&1

If you would rather not manage raw proxy credentials, the same probe through the FineData API is one POST to POST /api/v1/async/scrape — pin the country, set url to your echo endpoint, and read the IP out of the JSON envelope it returns:

API_URL = "https://api.finedata.ai/api/v1/async/scrape"
API_HEADERS = {"Authorization": "Bearer fd_your_api_key"}

def probe_exit_ip_via_api(requested_country="DE", session_id="geo-audit-de-01"):
    payload = {
        "url": "https://example.com",   # required -- the request fails without it.
                                         # Point this at your echo endpoint.
        "proxy_country": requested_country,
        "use_residential": True,
        "proxy_sticky": True,
        "session_id": session_id,
        "formats": ["text"],
        "raw_output": True,   # return the target's raw body, not extracted markdown
        "timeout": 30,
    }
    resp = requests.post(API_URL, headers=API_HEADERS, json=payload, timeout=60)
    # resp.status_code is the scrape API's status, not the echo target's.
    resp.raise_for_status()

    # raw_output=True: the target's raw response body rides inside the API's
    # JSON envelope, so parse the content key explicitly instead of scanning
    # resp.text -- the envelope's own metadata can contain IPs that are not
    # your exit.
    envelope = resp.json()
    return first_ipv4(envelope["data"]["content"])   # parse it out — never assume the IP

raw_output: true changes what comes back, and the semantics matter for the audit: with it set, data.content carries the target’s raw response body, byte for byte; without it you get extracted markdown, which can mangle an echo response. For an IP echo endpoint the body is the answer, so raw output is what you want. Everything around it is the API’s own envelope — job id and job status on the outside, the target’s URL and status code alongside the content — which is exactly why you read the data.content key explicitly rather than grepping resp.text: the envelope’s metadata can contain IPs that are not your exit, and you could not verify from resp.text alone whether the IP you found came from the target or the wrapper. A trimmed response looks like this:

{
  "job_id": "job_01JD8Z8K3M",
  "status": "completed",
  "data": {
    "url": "https://example.com",
    "status_code": 200,
    "content": "203.0.113.7\n"
  }
}

For longer scrapes the same endpoint accepts a callback_url, and job status is available at GET /api/v1/async/jobs/{job_id} — the audit logic below doesn’t care which delivery path you use.

Comparing Requested Country vs. Resolved Country and Flagging Mismatches

The comparison looks like a one-liner and isn’t. Three edge cases will pollute your score if you score them naively, so match_score returns 1 (hit), 0 (miss), or None (don’t count it):

import ipaddress

def match_score(requested_country, resolved_country, exit_ip=None):
    # Empty resolution: the database had no entry. That is our lookup's
    # gap, not the pool's failure -- counting it as a miss punishes the
    # wrong party.
    if not resolved_country or not requested_country:
        return None

    # Legacy allocations can resolve to "EU" instead of a member state.
    # Ambiguous for any EU request -- refuse to score it either way.
    if resolved_country == "EU":
        return None

    # Private or CGNAT exits mean the probe never really left your own
    # egress. Broken capture, not a geo failure.
    if exit_ip:
        try:
            addr = ipaddress.ip_address(exit_ip)
        except ValueError:
            return None
        if addr.is_private or addr in ipaddress.ip_network("100.64.0.0/10"):
            return None

    # ISO code alias: some databases still emit "UK" where ISO says "GB".
    # Normalize BOTH sides so the rule is symmetric -- requesting "UK" and
    # resolving "GB" scores the same as the reverse.
    alias = {"UK": "GB"}
    requested = alias.get(requested_country, requested_country)
    resolved = alias.get(resolved_country, resolved_country)

    return 1 if resolved == requested else 0

Note the alias rule normalizes both sides. Scoring GB-requested/UK-resolved as a hit while UK-requested/GB-resolved scored zero would be an asymmetry that silently penalizes one naming convention — the alias map is applied to requested and resolved alike.

Wire everything into a single probe function — fetch, exit-IP capture, resolution, scoring, one log row — so every audit request, real scrape or echo check, produces exactly one line:

def probe(url, requested_country, session_id=None):
    resp, ts, latency_ms = fetch_once(url)
    exit_ip = capture_exit_ip(resp, PROXY_CONFIG)
    resolved_country = resolve_country(exit_ip)
    row = {
        "ts": ts.isoformat(timespec="seconds"),
        "target": url,
        "requested_country": requested_country,
        "exit_ip": exit_ip,
        "resolved_country": resolved_country,
        "matched": match_score(requested_country, resolved_country, exit_ip),
        "latency_ms": latency_ms,
        "session_id": session_id,
        "status_code": resp.status_code,
    }
    append_log(row)
    return row

Because the gate encodes the country in the proxy username, each requested country gets its own config — the US row below comes from a second pass with REQUESTED_COUNTRY = "US". A run against https://example.com — five probes, three hits, one miss, one unresolved — lands like this (timestamps trimmed for display):

{"ts":"09:14:02Z","target":"https://example.com","requested_country":"DE","exit_ip":"198.51.100.23","resolved_country":"DE","matched":1,"latency_ms":640,"session_id":"geo-de-01","status_code":200}
{"ts":"09:14:04Z","target":"https://example.com","requested_country":"DE","exit_ip":"198.51.100.87","resolved_country":"DE","matched":1,"latency_ms":812,"session_id":"geo-de-01","status_code":200}
{"ts":"09:14:07Z","target":"https://example.com","requested_country":"US","exit_ip":"203.0.113.45","resolved_country":"US","matched":1,"latency_ms":703,"session_id":"geo-us-01","status_code":200}
{"ts":"09:14:09Z","target":"https://example.com","requested_country":"DE","exit_ip":"203.0.113.7","resolved_country":"FR","matched":0,"latency_ms":655,"session_id":"geo-de-02","status_code":200}
{"ts":"09:14:12Z","target":"https://example.com","requested_country":"DE","exit_ip":"192.0.2.14","resolved_country":null,"matched":null,"latency_ms":741,"session_id":"geo-de-02","status_code":200}

Reading the score. The number the whole audit exists to produce is three lines off the log file. Crucially, null rows are the “own lane” promised in the skeleton section: an unresolved lookup is a gap in your database, not a failure of the pool, so those rows enter neither the numerator nor the denominator.

rows = [json.loads(line) for line in open(LOG_PATH) if line.strip()]
scored = [r for r in rows if r["matched"] is not None]
accuracy = sum(r["matched"] for r in scored) / len(scored)   # 3 / 4 = 0.75

This run scored 75% — three hits out of four scored — with one request excluded as unresolved.

Triaging the mismatch. Row four is why the audit exists. Requested DE, resolved FR, status_code: 200, latency indistinguishable from the hits: every monitor you own is green, and the wrong country still served the page. That is precisely the French-pricing failure from the opening, now sitting in a log row with evidence attached. Work a miss in a fixed order:

1. Cross-check the exit IP against a second geolocation source. If a second database says DE, yours is stale, not the pool — refresh and rescore before you complain. This is the cross-check promised in the resolution section; it comes before everything else.
2. If both sources agree on FR, read the session_id column. Misses concentrated in one sticky session mean that session is pinned to a mislabeled exit — rotate the session and the misses leave with it. Misses scattered across sessions point at range-level mislabeling inside the pool, which only the provider can fix.
3. Escalate with rows, not screenshots. Timestamp, exit IP, requested country, resolved country, second-source confirmation — a JSONL row is a reproducible bug report. A dashboard average is an opinion.

The null lane has its own failure signature: if unresolved rows are more than a rounding error of your traffic, the instrument is broken — usually a stale database, or an exit-IP capture that has started picking up private ranges — and you fix the audit before you judge the pool. Finally, give the fraction room to mean something: a few hundred probes per requested country, spread across times of day, before you accept or reject a pool. Five rows demonstrate the mechanics; the log is what certifies the pool. And the next time a storefront serves French pricing to a German exit, this is the file you open — one row, two labels, and no dashboard between you and what actually happened.