You purchased residential proxies, set a fresh Chrome User-Agent, and the site still returns a 403 on the very first request. Sound familiar? The issue isn't with the IP or the headers. You've been detected even before the server read a single HTTP header β through the TLS handshake. In 2026, this is the number one detection vector, and a regular requests fails it automatically. Let's break down how this works and how to fix it with just a few lines of code using curl_cffi.
What's happening: you're being caught by the TLS handshake
When a client establishes an HTTPS connection, it first sends a ClientHello packet β even before any HTTP. This packet includes: the TLS version, a list of supported cipher suites, TLS extensions (SNI, ALPN, supported_groups), elliptic curves, and point formats. The order and composition of these fields vary among different clients significantly β and based on this, the client can be identified before it utters a single word.
From these fields, a fingerprint is calculated. JA3 (a standard from 2017) takes a string in the form of TLSVersion,Ciphers,Extensions,EllipticCurves,ECPointFormats and hashes it with MD5, resulting in a 32-character signature. The problem with JA3 is that since January 2023, Chrome randomizes the order of extensions β 16 extensions yield 16! (over 20 trillion) combinations, and the same browser produces different JA3 signatures.
Therefore, the industry has transitioned to JA4 (FoxIO, widespread adoption in 2024β2025). JA4 sorts the extension codes by their hex value before hashing β Chrome's randomization no longer breaks it. The hash is a truncated SHA-256, human-readable, and in a three-part format (a_b_c), which includes ALPN and QUIC/HTTP3 support. For example, Chrome 124 gives t13d1516h2 (15 ciphers, 16 extensions, ALPN h2), while plain Python requests yields t13d1715h2. For anti-bot systems, the second signature is a direct marker indicating "this is a script."
Why in 2026 this is essential
JA4 detection is embedded in all major vendors: Cloudflare checks the fingerprint against allowlists, Akamai adds a separate hash for HTTP/2 SETTINGS frames, and DataDome compares it to a database of known bots. The logic is simple and lethal: if you send User-Agent: Chrome 131, but the TLS fingerprint screams "urllib3/OpenSSL" β that's a desynchronization, and you are blocked instantly. No proxy can save you: a perfect residential IP with a Python requests fingerprint still loses.
This is precisely why the combination of "proxy + fingerprint spoofing" became basic hygiene for scraping in 2026, rather than an option for the advanced.
Solution: curl_cffi in 5 minutes
curl_cffi is a Python wrapper around curl-impersonate (a modified curl built with BoringSSL from Chrome or NSS from Firefox instead of OpenSSL). It reproduces authentic browser handshakes while having an API almost identical to the familiar requests.
Step 1. Installation. The curl-impersonate binaries for Windows/macOS/Linux are pulled automatically:
pip install curl-cffi
Step 2. Basic request. Change the import and add one parameter:
from curl_cffi import requests
resp = requests.get("https://target.com/", impersonate="chrome")
print(resp.status_code)
print(resp.http_version) # HTTP/2 β just like a real browser
The single line impersonate="chrome" spoofs four layers at once: the TLS fingerprint (JA3/JA4), the HTTP version (HTTP/2 instead of HTTP/1.1), the order of headers, and ALPN negotiations.
Step 3. Always use the generic alias, not a pinned version. Write impersonate="chrome" (or "safari", "safari_ios") β the alias automatically resolves to the latest profile. A hardcoded impersonate="chrome124" will become outdated: Chrome updates approximately every 4 weeks, and an old profile will itself become an anomaly. Reliable targets are Chrome, Edge, and Safari/iOS (profiles from chrome99 to chrome131, safari15β18).
Step 4. Proxies and sessions. For real scraping, maintain state in a session and attach proxies. A residential or mobile IP is essential here β a data center is flagged separately from TLS:
from curl_cffi import requests
session = requests.Session(impersonate="chrome")
headers = {
"Accept-Language": "en-US,en;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Referer": "https://www.google.com/",
}
proxies = {
"http": "http://user:pass@proxy-host:port",
"https": "http://user:pass@proxy-host:port",
}
resp = session.get("https://target.com", headers=headers, proxies=proxies)
Step 5. Asynchronous for volume. Unlike requests, curl_cffi comes with async and HTTP/2 out of the box:
import asyncio
from curl_cffi.requests import AsyncSession
async def fetch(session, url):
r = await session.get(url, impersonate="chrome")
return r.status_code
async def main(urls):
async with AsyncSession() as session:
return await asyncio.gather(*[fetch(session, u) for u in urls])
asyncio.run(main(["https://target.com"] * 20))
Check your fingerprint β don't guess
Before sending live traffic, ensure that the spoofing is genuinely working. Send a request to public verifiers and compare JA4 with a benchmark browser:
- tls.peet.ws β returns JA3, JA4, Akamai fingerprint, and HTTP/2 frames in JSON. Request it via
curl_cffiand through real Chrome, and compare the hashes. - ja4db.com β a database of known JA4s, helps you understand who you resemble.
- browserleaks.com/tls and the JA3/JA4 tool from Scrapfly β detailed breakdown of fields.
In staging, it's convenient to place mitmproxy between the scraper and the target to monitor the actual JA4 hash of each request.
Diagnostics: still getting 403/429
If the fingerprint is correct but blocks remain β go through the checklist from common to rare:
- Data center IP. Reason #1. Switch to residential proxies or mobile ones β why just a fingerprint is insufficient is thoroughly explained in the article about detecting residential proxies through IP Intelligence.
- Outdated profile.
pip install -U curl-cffiand the generic alias"chrome". - Too high rate. Add random pauses of 1β3 seconds between requests.
- Naked headers. Always send
Accept-Language,Accept-Encoding,Refererβ their absence is also an anomaly. - Desynchronization of session and IP. Rule: one session β one IP for its entire lifetime.
- Status 200 β success. Check the response body: a page with CAPTCHA may lie under code 200.
Where curl_cffi hits a wall
curl_cffi closes the network layer β and that's it. It does not execute JavaScript. Therefore, it is powerless against JS challenges: Cloudflare Turnstile, the "Checking your browserβ¦" page (IUAM), the cf_clearance cookie set by the script after verification β all of this requires a real browser environment. Why in 2026 CAPTCHA solvers have almost stopped working against such preventive systems has been discussed in a separate analysis of CAPTCHA bypass.
What to do when you hit a JS wall:
- Hybrid. Use Playwright or Nodriver to run the challenge and obtain
cf_clearance, then pass the cookie to fastcurl_cffifor the bulk of requests β this way, you pay for the heavy browser only once. - Solver services (CapSolver, 2Captcha) for automatic token issuance.
- Managed scraping APIs if you don't want to maintain infrastructure.
And remember about thread safety: each thread should have its own session. Pin the version of curl-cffi in requirements.txt and review profiles every 6β12 weeks when browsers update.
Alternatives to curl_cffi
- tls-client β a wrapper around the Go library based on uTLS, with profiles (
chrome_124,safari_ios_17) and a flagrandom_tls_extension_order=True. Flexible fine-tuning of the fingerprint. - primp β a Rust client that allows independent specification of
impersonate_osand provides higher throughput; downside β the API does not fully matchrequestsand the library is younger.
What kind of proxy is needed and why
Fingerprint spoofing and proxies solve different halves of the same problem: curl_cffi addresses the question of "what does the connection look like," while proxies address "where is it coming from." An anti-bot checks both signals independently, so an ideal JA4 with a black data center ASN is useless. For secure targets (marketplaces, social networks, travel aggregators), use residential or mobile proxies: they have a clean operator origin, and mobile ones also hide behind the CGNAT "crowd effect." Reserve data centers for non-sensitive purposes and high volume.
Conclusion
In 2026, scraping is a game of identities, not just IPs. Plain requests presents itself as a script at the TLS handshake level and loses before the first header. Replacing the import with curl_cffi and impersonate="chrome" eliminates this failure in five minutes, but it only works in conjunction with a clean residential or mobile IP and with an understanding of the boundary: network layer β yes, JavaScript challenges β no. Build your stack honestly: correct fingerprint, correct proxy, hybrid with a browser where there is a JS wall β and 403 on the first request will be a thing of the past.
