Scarnox · Blog

Browser fingerprinting explained for scrapers

Browser fingerprinting is the practice of identifying a visitor by the combined traits of their browser, not by a cookie or an account. Sites use it to spot returning visitors, to detect fraud, and to recognise automation. If you scrape the web, you will meet it. This guide explains what is measured, why a single mismatch causes trouble, and how to keep one consistent, well-behaved profile for collecting public data. It also explains the limits of what fingerprint tuning can and should do.

What a fingerprint is made of

No single value identifies a browser on its own, but many small values together often do. Common inputs include:

User agent and client hints. The browser name, version and operating system the browser reports.
Screen and window geometry. Screen size, colour depth, device pixel ratio and the size of the viewport.
Language and timezone. The Accept-Language header, navigator.languages and the timezone the browser reports.
Installed fonts. The fonts the page can detect, which differ between operating systems and installed software.
Canvas, WebGL and audio output. Small differences in how the graphics or audio stack draws or processes a test image or sound.
Network-level traits. The TLS handshake, HTTP/2 settings and the order of request headers. These are visible to the server even before any JavaScript runs.

Fingerprinting scripts combine these into a hash or a score. The more signals that are stable and unusual, the more identifying the result.

Why a mismatch stands out

A real browser is internally consistent. Its operating system matches its fonts, its timezone matches its language setting, and its screen size matches the device it runs on. Scrapers often break that consistency without meaning to:

A user agent copied from a desktop browser, sent by a server with no fonts installed.
A German Accept-Language header on a request whose IP address is in another country, with a timezone that matches neither.
A headless environment that reports a window size no real monitor uses.
A brand-new profile on every run, so the cookies and history never look like a returning visitor.

Each mismatch is a small signal. Together they can move a request from "ordinary visitor" to "needs a closer look".

Step 1: measure your own setup

You can see what a page learns about your client by loading a fingerprint test page that you control, or a diagnostic page you run locally. Look at the values and ask whether they describe one coherent machine. Here is a minimal Playwright example that prints a few of them:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    context = browser.new_context(locale="en-GB", timezone_id="Europe/London")
    page = context.new_page()
    page.goto("https://example.com")  # replace with a page you own
    print(page.evaluate("""() => ({
        language: navigator.language,
        timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
        platform: navigator.platform,
        viewport: [innerWidth, innerHeight]
    })"""))
    browser.close()

Step 2: keep one identity per profile

The most useful habit is to treat each scraping identity as one profile. Give it a single set of traits and keep them for its whole life:

One browser build, one user agent and one operating system.
A locale, language and timezone that match the region the traffic comes from.
A viewport and device scale factor that match a real screen.
A persistent cookie jar, so the profile behaves like a returning visitor.

Changing one of these for every request is more suspicious than keeping it steady. Consistency is what makes a profile look ordinary.

Step 3: pace the profile like a visitor

Fingerprinting is only one input. Sites also watch how fast one profile moves through the site. A profile that loads two hundred product pages in a minute is not a human, whatever its fingerprint says. Spread requests out, cache results, and stop when a page indicates you should. The scrape without getting blocked checklist covers pacing in more detail.

Step 4: know the limits

Fingerprint tuning cannot make an automated profile look like a person who is not there. Sites that rely on fingerprinting usually combine it with rate limits, account checks and, sometimes, a captcha. When a captcha appears, you need a token from a solver rather than more fingerprint tweaks. The live captcha type at Scarnox is PopularCaptcha; the PopularCaptcha Python guide shows the request. For the wider picture of which widgets exist, read captcha types explained.

More importantly, fingerprint evasion is not a licence to ignore a site's rules. If the owner has restricted automated access, the right move is to stop, not to make your profile harder to notice. Fingerprinting is also a privacy topic. Some sites use it to track people across visits, and that is one more reason to collect only what your project needs.

Scrape responsibly

Read robots.txt and the terms before you scrape a site.
Use fewer, slower requests, and cache what you have already collected.
Do not build profiles to impersonate real people or to get around a block.
Do not store personal data you do not need, and handle what you do store with care.

Where to go next

If your target sits behind a browser-level check, start with headless browser detection, which covers the signals specific to automated browsers. Pages that need JavaScript to show their content are covered in scraping JavaScript websites.

Start with free trial credit, no card needed: create an account and try the PopularCaptcha flow against a page you own. For current rates, see the pricing page.