Classic combinations of Selenium and Puppeteer are increasingly stumbling upon anti-bot systems like Cloudflare, DataDome, and PerimeterX. New frameworks are emerging — Scrapling, Botasaurus, and nodriver, which were originally designed with a focus on bypassing detection. However, even the most advanced framework is useless without properly configured proxies. In this article, we will discuss how to connect proxies in each of the three tools, which mistakes kill effectiveness, and how to choose the type of IP for a specific parsing task.
How Scrapling, Botasaurus, and nodriver differ from Selenium
All three frameworks emerged in response to the fact that Selenium and standard Puppeteer became too easily detectable. Anti-bot systems check dozens of parameters: CDP protocol, navigator.webdriver, canvas rendering, event timings — and classic drivers leave characteristic traces in almost all of them.
Scrapling is a Python library that combines adaptive HTML parsing with the ability to emulate a browser through StealthyFetcher based on Camoufox (a fork of Firefox with a patched fingerprint). It is suitable for tasks where fast parsing with minimal resource consumption is needed, but the website's protection actively checks the TLS fingerprint.
Botasaurus is a framework based on a modified Chromium (undetected-chromedriver under the hood), designed for mass parallel scraping with automatic browser profile management and built-in support for solving CAPTCHAs through third-party services.
nodriver is the successor to undetected-chromedriver from the same author, who completely abandoned Selenium WebDriver and works directly through CDP (Chrome DevTools Protocol), which removes a significant portion of detectable automation artifacts.
A common feature of all three is that they reduce detection at the browser fingerprint level, but do not solve the IP reputation problem. If requests come from a data center IP that is already flagged in the blacklists of DataDome or Cloudflare, no browser patch will save you from CAPTCHA or a 403 error.
Setting up proxies in Scrapling
Scrapling supports proxies in both fast HTTP mode (Fetcher) and browser mode (StealthyFetcher). For static parsing without JS rendering, the proxy is passed directly in the request parameters:
from scrapling.fetchers import Fetcher
response = Fetcher.get(
"https://example.com",
proxy="http://user:[email protected]:9000"
)
print(response.status)
For the mode with a full browser (StealthyFetcher on Camoufox), the proxy is passed through a dictionary with separate fields for server, username, and password — this is important because Camoufox handles proxy authorization differently than standard Chromium:
from scrapling.fetchers import StealthyFetcher
response = StealthyFetcher.fetch(
"https://example.com",
proxy={
"server": "gate.proxycove.com:9000",
"username": "user",
"password": "pass"
},
headless=True
)
print(response.status)
A common mistake is passing the proxy as a single string in browser mode, as in HTTP requests. In this case, Camoufox either ignores the authorization or fails with a timeout when connecting to the proxy server. If you see the error ERR_NO_SUPPORTED_PROXIES — check the format specifically as a dictionary with separate fields.
Setting up proxies in Botasaurus
Botasaurus uses the @browser
decorator for driver configuration, and the proxy is passed as a parameter of this decorator or directly in the launch function:
from botasaurus.browser import browser, Driver
@browser(
proxy="http://user:[email protected]:9000"
)
def scrape_data(driver: Driver, data):
driver.get("https://example.com")
return driver.page_html
scrape_data()
If you need to connect different proxies for each parallel task (which is critical for mass scraping through Botasaurus), it is better to pass the proxy dynamically through a data list, rather than hardcoding it in the decorator:
from botasaurus.browser import browser, Driver
@browser
def scrape_with_proxy(driver: Driver, data):
driver.config.proxy = data["proxy"]
driver.get(data["url"])
return driver.page_html
tasks = [
{"url": "https://example.com/page1", "proxy": "http://user:pass@ip1:port"},
{"url": "https://example.com/page2", "proxy": "http://user:pass@ip2:port"},
]
scrape_with_proxy(tasks)
Botasaurus excels at parallelizing tasks through multiprocessing, so for mass scraping, it is important to use proxies with a sufficient pool of IPs — otherwise, several threads will hit the same address, which instantly reveals automation to the anti-bot system based on request frequency patterns.
Setting up proxies in nodriver
nodriver works through a direct connection to CDP without a Selenium layer, so proxies are configured through browser launch arguments, not through a separate API. For proxies without authentication, it's straightforward:
import nodriver as uc
import asyncio
async def main():
browser = await uc.start(
browser_args=["--proxy-server=http://gate.proxycove.com:9000"]
)
page = await browser.get("https://example.com")
await asyncio.sleep(3)
browser.stop()
asyncio.run(main())
The problem is that Chromium does not support username/password in proxies through --proxy-server directly.
For authenticated proxies in nodriver, you either need to set up a local proxy tunnel without authentication
(for example, through px or 3proxy), or handle the CDP event Fetch.authRequired:
import nodriver as uc
import asyncio
async def main():
browser = await uc.start(
browser_args=["--proxy-server=http://gate.proxycove.com:9000"]
)
page = await browser.get("https://example.com")
async def auth_handler(event):
await page.send(uc.cdp.fetch.continue_with_auth(
request_id=event.request_id,
auth_challenge_response=uc.cdp.fetch.AuthChallengeResponse(
response="ProvideCredentials",
username="user",
password="pass"
)
))
page.add_handler(uc.cdp.fetch.AuthRequired, auth_handler)
await asyncio.sleep(5)
browser.stop()
asyncio.run(main())
This is the least obvious point among all three frameworks — many developers spend hours trying to understand why authorization simply does not work, unaware of the Fetch domain in CDP.
Proxy rotation and fighting anti-bots
Even a perfectly configured browser fingerprint won't save you if more than 50-100 requests per minute are sent from one IP to a protected domain. DataDome and Cloudflare analyze the frequency of requests from a specific address and the timing patterns between actions — too uniform intervals are also suspicious.
The working rotation scheme for all three frameworks is built the same way:
- Use a gateway with automatic IP rotation at the provider level — there is no need to write your own logic for changing addresses for each request.
- For session tasks (login, multi-page navigation), fix the IP for the session through a sticky parameter in the connection string.
- For simple parsing of static pages, rotate IPs for each request or every 5-10 requests.
- Add random delays between requests (1-4 seconds) — this reduces detection based on timing patterns.
An example of a sticky session in the connection string, where the session-id retains the same outgoing IP for 10 minutes:
proxy = "http://user-session-abc123:[email protected]:9000"
For Botasaurus, this scheme is especially useful because during parallel scraping, each process can receive its unique session-id, and you get a stable pool of different IPs without manually managing a list of addresses.
Which type of proxy to choose for each framework
The type of proxy affects the success of parsing as much as the choice of framework. Below is a comparison by tasks:
| Task | Recommended type of proxy | Why |
|---|---|---|
| Parsing sites with Cloudflare/DataDome | Residential proxies | Real IPs of home users rarely end up in anti-bot blacklists |
| Mass scraping without strict protection | Data center proxies | Higher speed, lower cost for large volumes of requests |
| Emulating a real user on mobile sites | Mobile proxies | CGNAT addresses of telecom operators have the highest level of trust with anti-bots |
For Scrapling in StealthyFetcher mode, residential proxies provide the greatest effect because Camoufox already solves the browser fingerprint problem — it remains only to avoid detection by IP. For Botasaurus during mass parallel scraping, data center proxies are often optimal in terms of speed/price ratio, if the target site does not use advanced protection like DataDome.
Common mistakes when linking proxies and browser automation
In practice, most scraping failures are related not to the framework itself, but to incorrect proxy configuration. Here is a list of the most common mistakes:
- The same IP for all parallel Botasaurus threads. The anti-bot sees 20 simultaneous sessions from one address and blocks it within minutes.
- Passing the proxy as a string in Scrapling's browser mode. Camoufox requires a dictionary with separated fields — otherwise, authorization fails.
- Ignoring the CDP Fetch domain in nodriver. Without an authRequired handler, the authenticated proxy simply won't connect, and the browser will hang on loading.
- No rotation during session actions. Changing IP in the middle of an authorized session instantly reveals automation — use sticky sessions.
- Mismatch between proxy geolocation and browser locale. An IP from Germany with Accept-Language: ru headers — a classic sign of a bot for fraud scoring systems.
- Too aggressive request frequency even with rotation. IP rotation does not negate the need to pause between requests — behavior patterns are also analyzed.
It is also important to check for DNS leaks when working through proxies in browser mode — sometimes the browser resolves domains through the system DNS instead of the proxy tunnel, creating a mismatch between the request IP and the resolve IP, which can easily be detected by security systems.
Conclusion
Scrapling, Botasaurus, and nodriver address the detection problem at the browser fingerprint level much better than classic Selenium, but none of them diminish the importance of quality proxies. Proper configuration of authorization (especially in nodriver through CDP), smart IP rotation, and choosing the right type of proxy for a specific anti-bot system — these are the three factors that determine the stability of parsing in the long term.
If you are scraping sites with advanced protection like Cloudflare or DataDome, start with residential proxies — they provide the highest percentage of successful requests precisely due to IP reputation. For mass tasks without strict protection, data center proxies remain the optimal choice in terms of speed and cost for large volumes.