A classic situation: a script for monitoring prices on Wildberries or Ozon works perfectly on a home laptop, but after being transferred to a VPS, it starts receiving 403 errors, captchas, or instant IP bans. The developer changes headers, adds delays—but the result remains the same. The problem is almost never in the parser's code itself, but in the environment from which it makes requests. We analyze 7 specific reasons and what to change to ensure the parser works reliably on the server.
Why does everything work locally, but there's a ban on the server?
When you run the parser from a home computer, the site sees the request from a regular residential IP address of your provider, from a familiar region, with a real browser environment if you are using Selenium or Playwright with a real profile. As soon as the same script moves to a VPS in Germany, the Netherlands, or the USA, the picture changes completely: the IP belongs to a data center, the TLS fingerprint may differ due to different library versions, the server's time zone does not match the IP's geolocation, and the request frequency sharply increases because the server operates 24/7 without breaks.
Anti-bot systems of Wildberries, Ozon, Avito, and most major marketplaces no longer only look at the User-Agent. They analyze a combination of dozens of signals: IP type, speed and regularity of requests, on-page behavior, compliance of headers and TLS parameters, presence of cookies, and session history. The local machine randomly passes checks on most points, while the server fails almost all of them. Below is a detailed analysis of each reason.
Reason 1: Data center IP instead of residential IP
This is reason #1 in 80% of cases. The IP addresses of VPS and cloud servers (AWS, DigitalOcean, Hetzner, regular VDS hosting) are listed in data center databases—the ASNs of these providers are publicly known and used by anti-bot systems for instant traffic filtering. Marketplaces primarily use such lists because 95% of automated parsing comes from server IPs.
The solution is to use IPs that visually do not differ from a regular internet user. For parsing Wildberries, Ozon, and Avito, residential proxies are the best fit: these are real IP addresses from home providers, issued to regular subscribers. Anti-bot systems see such requests as traffic from a live user, not from a server in a data center, which removes a large portion of blocks immediately.
Reason 2: No IP rotation and request rate limit
On a local machine, you make 20–50 requests manually during tests, and the site does not notice. On the server, the script is run via cron every 5 minutes and processes thousands of product cards consecutively from one IP. Such a pattern is a direct signal for the anti-bot system: a real person cannot open 3000 catalog pages in an hour without a single pause.
You need to implement IP rotation on the proxy pool and limit the number of requests to one address in a given time frame. A practical rule: no more than 30–60 requests from one IP per minute for product cards, with automatic address change after each batch of requests. Here’s an example of setting up rotation in Python through a proxy pool:
import requests
proxies_pool = [
"http://user:[email protected]:9000",
"http://user:[email protected]:9001",
"http://user:[email protected]:9002",
]
def get_page(url, session_id):
proxy = proxies_pool[session_id % len(proxies_pool)]
resp = requests.get(
url,
proxies={"http": proxy, "https": proxy},
headers={"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"},
timeout=10
)
return resp.textWhen collecting a large volume of cards per day, it is more convenient to use proxies with automatic IP rotation per request or timer—this eliminates the need to maintain a list of addresses manually.
Reason 3: Headers and User-Agent do not resemble a browser
Many parsers using requests or aiohttp send requests with a minimal set of headers or with the standard User-Agent of the library, which immediately reveals the script (for example, python-requests/2.31.0). On a local machine through a browser, the set of headers is complete: Accept, Accept-Language, Accept-Encoding, Sec-Ch-Ua, Referer, and others—this combination looks natural.
You need to copy the full set of headers from a real browser, including the order in which they are sent—some anti-bot systems check even this. Additionally, it is important to rotate the User-Agent synchronously with the TLS fingerprint version (see the next point), otherwise, the mismatch between the browser header and the real TLS client will become a new bot signal.
Reason 4: TLS/JA3 fingerprint reveals the script
This is a less known but extremely common reason for bans specifically on the server. The requests, urllib, and aiohttp libraries use their own implementation of the TLS handshake, which differs from the implementation in Chrome or Firefox. Anti-bot systems calculate the JA3/JA4 fingerprint of the TLS connection—and it does not resemble the fingerprint of a real browser at all, even if the headers are copied perfectly.
The solution is to use libraries that emulate the browser's TLS fingerprint (for example, curl_cffi, tls-client in Python, or a full-fledged headless browser based on Chromium via Playwright/Puppeteer). The second option is to work not through a pure HTTP client but through a managed browser engine in conjunction with an anti-detect tool, where TLS and headers are formed by a real browser core, not an emulation of a library.
Reason 5: Time zone, locale, and DNS servers
If the script emulates a browser through Selenium or Playwright, the anti-bot system may check the system time zone, interface language, DNS resolver, and even WebRTC leak of the server's real IP. A VPS in a Frankfurt data center with a system time zone of UTC and a hoster’s DNS provider, while using a proxy with an IP from Moscow, creates a clear mismatch in geodata—this is one of the most reliable signals for detection.
All environmental parameters—the time zone, browser language, DNS, geolocation via WebRTC—must match the region of the IP address used for the request. Anti-detect browsers were created specifically to solve this problem: Dolphin Anty, AdsPower, Multilogin, Octo Browser, and GoLogin allow you to set up a separate "browser profile" for each proxy, where the time zone, locale, screen resolution, and WebRTC are automatically adjusted to the geolocation of the IP.
Reason 6: Request pattern is too "robotic"
A person scrolls through the catalog with varying pauses, clicks on random products, sometimes goes back, scrolls the page unevenly. A server script typically makes requests at equal intervals (for example, strictly every 2 seconds) and only accesses the necessary URLs without "noise" around—without loading images, scripts, or visiting the main page before the product card.
What to change: add random delays (not fixed 2 seconds, but a random range from 1.5 to 6 seconds), occasionally visit intermediate pages (category → card, not a direct request to the API), emulate scrolling and mouse movements when working through a headless browser. This increases the data collection time but sharply reduces the number of bans.
Reason 7: Sessions and cookies are not preserved between requests
Often, the parser on the server creates a new requests session for each request—without cookies, without a saved authorization token, without a visit history. Marketplaces like Wildberries and Ozon issue temporary cookies and tokens on the first visit, and subsequent requests without them look suspicious, as if each request is made by a new anonymous visitor.
The correct scheme: one session (requests.Session() or browser context) for one IP from the proxy pool, with cookies preserved throughout the entire series of requests to that IP. When changing proxies, you need to start a new session with clean cookies, simulating a new user, rather than continuing to use old cookies with a new IP—this also creates a mismatch and triggers a block.
Checklist: what to change in order
If the parser is consistently banned on the server but works locally, check the changes in this order—this way you will find the reason faster:
| Step | What to check | What to change |
|---|---|---|
| 1 | Type of server IP | Switch to residential proxies instead of direct hosting IP |
| 2 | Request frequency | Implement IP rotation and limit requests to an address |
| 3 | Request headers | Copy the full set of headers from a real browser |
| 4 | TLS fingerprint | Use curl_cffi / headless browser instead of pure requests |
| 5 | Time zone and locale | Set up a profile in Dolphin Anty / AdsPower for the IP region |
| 6 | Behavior pattern | Randomize delays, add intermediate pages |
| 7 | Sessions and cookies | Bind one session to one IP for the entire request cycle |
For high-frequency parsing of Wildberries and Ozon catalogs, where speed is crucial for browsing thousands of pages, it is often effective to combine two types of proxies: data center proxies for rough technical requests (availability checks, status codes) and residential proxies for final data collection from cards, where masking as a real user is important. For mobile applications of marketplaces and Avito, sometimes mobile proxies are more effective, as they are less likely to end up on automatic blocking lists by ASN.
Conclusion
The banning of a parser on the server while the local version works is almost always related not to the logic of the script but to the environment: type of IP, TLS fingerprint, headers, time zone, request pattern, and session management. By checking each of the 7 reasons in order—from the most common (data center IP) to the least noticeable (mismatch of time zone and IP region)—you can restore stable operation of the parser without changing the core business logic of data collection.
If you are collecting prices and stock levels on Wildberries, Ozon, or Avito in industrial volumes, start by replacing the IP: try residential proxies instead of the standard VPS address—in most cases, this eliminates up to 70% of blocks even before you start configuring headers and TLS fingerprints.