Dynamic pricing has become the standard on marketplaces — prices change 10-20 times a day depending on demand, stock levels, and competitors' actions. If you are a seller on Wildberries, Ozon, or Avito, you know: the one who learns first about a competitor's price drop retains their position in the search results. However, all marketplaces strictly block mass scraping from a single IP. In this article, we will explore how to set up automatic price monitoring through proxies to collect data 24/7 without the risk of blocking your account or IP address.
Why Marketplaces Block Price Scraping and How It Works
All major marketplaces use multi-layered protection against automated data collection. The reason is simple: mass scraping creates a load on servers and gives a competitive advantage to those who can collect data. Wildberries, for example, records up to 300,000 price changes a day — and does not want this data to be obtained by competitors for free.
Here’s how a typical anti-scraping protection system works:
- Tracking Request Frequency from a Single IP — if you open 50+ product pages in a minute, the system understands that it’s not a human. A regular buyer views 3-5 products per minute, while a scraper views 100-500.
- Analysis of User-Agent and Headers — requests without correct browser headers are immediately blacklisted. Many beginners use standard Python requests libraries without setting headers — this is the first thing that detection systems catch.
- JavaScript and Cookies Check — modern marketplaces use Cloudflare, DataDome, and their own anti-bot systems. They check for JavaScript execution, the presence of session cookies, mouse movement, and scrolling behavior.
- Geographical Binding of IP — if you scrape Russian Ozon with an IP from Germany or the USA, it looks suspicious. The system may show you outdated prices or completely block access.
- Blocking Data Centers — most marketplaces maintain databases of IP addresses from cloud providers (AWS, Google Cloud, Hetzner) and automatically restrict requests from them.
The result of blocking can vary: from temporary rate limiting to a complete IP ban for several weeks. If you scrape through a seller account, you risk getting your personal account blocked, which is critical for business.
Real Case: A seller on Wildberries set up scraping competitor prices through a regular VPS server. In 2 hours, he collected data on 500 products. A day later, he discovered that his IP was completely blocked — even accessing the site as a regular buyer became impossible. He had to wait 2 weeks for the block to be lifted automatically.
What Types of Proxies Are Suitable for Price Monitoring: A Comparison
For monitoring dynamic pricing, three types of proxies are suitable, each with its own pros and cons. The choice depends on your budget, scraping volume, and geographical requirements.
| Proxy Type | Advantages | Disadvantages | For Which Tasks |
|---|---|---|---|
| Residential Proxies | Real IPs of home users, minimal risk of bans, support for any geography, bypassing Cloudflare | More expensive than other types, speed may vary | Scraping Wildberries, Ozon, sites with strict protection. Monitoring regional prices. |
| Mobile Proxies | IPs from mobile operators, one IP is used by thousands of users — almost impossible to block, high trust score | The most expensive, limited geography, periodic IP changes by the operator | Scraping through mobile apps of marketplaces, bypassing the strictest protection |
| Data Center Proxies | High speed, low cost, stable connection, unlimited traffic | Easily detected, many marketplaces preemptively block data centers | Scraping sites without serious protection, Avito, small online stores |
Practical Recommendation for Selection:
- For Wildberries and Ozon — only residential or mobile proxies. These platforms use Cloudflare and their own anti-bot systems, data centers are blocked in 90% of cases.
- For Yandex.Market — residential proxies with Russian geography, the system checks the compliance of IP and search region.
- For Avito — data centers can be used, but with rotation and delays between requests. Protection is weaker than that of marketplaces.
- For AliExpress and international platforms — residential proxies from the required country, many show different prices for different regions.
If your budget is limited — start with a pool of 20-30 residential proxies with rotation. This will be enough to monitor 500-1000 products a day without blocks.
Protection Features of Wildberries, Ozon, Yandex.Market, and Avito
Each platform uses its own combination of protective mechanisms. Understanding these features helps to set up the scraper correctly the first time.
Wildberries
The strictest protection among Russian marketplaces. They use Cloudflare with "I'm Under Attack" mode enabled, which checks for JavaScript challenges before granting access to content. Additionally, they track:
- Browser fingerprint — canvas, WebGL, fonts, screen resolution
- TLS fingerprint — protocol version and set of cipher suites
- Behavioral factors — scrolling speed, mouse movements, time on page
- Session history — if you immediately start opening product pages without viewing the main page and categories, it looks suspicious
What Works: Headless browsers (Puppeteer, Playwright) with residential proxies, emulating real user behavior, delays of 5-15 seconds between requests, rotating User-Agent and proxies after every 20-30 requests.
Ozon
They use their own anti-bot system plus DataDome. A feature: prices may differ depending on the delivery city — the system determines the region by IP and shows corresponding prices considering logistics.
Ozon actively blocks known ranges of data center IPs. When scraping through a regular VPS, you will get a captcha or a 403 error in 80% of cases. They also track cookies — if you make requests without saving the session between requests, this is detected.
What Works: Residential proxies with Russian geography, mandatory saving of cookies between requests, simulating adding products to favorites (this creates a "history" for the user), delays of 3-10 seconds.
Yandex.Market
Average level of protection. The main feature is strict regional binding. If you scrape prices for Moscow with an IP from Vladivostok, the system will either show prices for Vladivostok or completely block access. For correct monitoring, proxies from the region whose prices you are tracking are required.
Yandex also uses its own anti-fraud system that analyzes the sequence of actions. It looks suspicious if: you open product pages via direct links without searching, have identical intervals between requests (for example, strictly every 5 seconds), or do not make requests to static content (images, styles).
What Works: Residential proxies from the required region, emulating a full cycle: search → product list → product page, random delays of 4-12 seconds, periodic proxy changes.
Avito
The most lenient protection among the listed platforms. The main protection is a limit on the number of ad views from one IP (about 100-150 ads per hour). After exceeding the limit, a captcha or temporary block for 1-2 hours is shown.
Avito also checks for cookies and basic headers but does not use complex JavaScript challenges. Data centers work, but with limitations — rotation and moderate request frequency are needed.
What Works: Even data centers with rotation, delays of 2-5 seconds between ads, correct browser headers, saving cookies. For large volumes — residential proxies.
Setting Up Proxy Rotation for Round-the-Clock Scraping
Proper proxy rotation is a key factor for stable scraping without blocks. There are three main rotation strategies, each suitable for different scenarios.
Strategy 1: Rotation by Number of Requests
Essence: change proxies after a certain number of requests (for example, after every 20-50 requests). This is the simplest and most effective method for most tasks.
Recommended Values for Different Platforms:
- Wildberries: 15-25 requests per proxy, then change
- Ozon: 20-30 requests
- Yandex.Market: 30-50 requests
- Avito: 50-100 requests
An important nuance: add randomization. Do not change proxies strictly after 20 requests — do it randomly after 18-23 requests. This makes the pattern less predictable for detection systems.
Strategy 2: Time-Based Rotation
Change proxies every N minutes regardless of the number of requests. Suitable for tasks with unpredictable loads — for example, real-time price monitoring, where the number of updates may change sharply.
Recommended Intervals: 5-15 minutes for strict platforms (Wildberries, Ozon), 15-30 minutes for more lenient ones (Avito, small stores).
Strategy 3: Sticky Sessions
Use the same proxy for all requests within one "user session." For example: you emulate a buyer who logged into Ozon, searched for "laptops," opened 5 product pages, and added one to the cart. All these actions go through one proxy with cookies saved.
After the session ends (after 3-10 minutes) — change the proxy and start a new session. This is the most "human" pattern but requires more proxies and is more complex to implement.
When to Use: Scraping platforms with very strict protection, where simple rotation does not work. Also suitable for collecting data through seller personal accounts.
Practical Advice:
Combine strategies. For example: use sticky sessions for 20-30 requests, and after the session ends, change the proxy. Add a time limit — if the session lasts more than 10 minutes, forcibly change the proxy. This hybrid approach provides maximum protection against detection.
Ready-Made Tools for Price Monitoring with Proxy Support
If you don't want to write a scraper from scratch, there are ready-made solutions with built-in proxy support. Let’s look at the most popular tools for price monitoring on Russian and international platforms.
For Russian Marketplaces
1. Mpstats (mpstats.io)
A specialized service for analytics on Wildberries and Ozon. Collects data on sales, stock levels, and competitor prices. There is an API for integration with your own systems. The service operates through its own proxy infrastructure, so you don't need to set anything up additionally.
Cons: high cost (from 15,000 rub/month for full access), you cannot use your own proxies. Suitable for large sellers with a turnover of 1-2 million rub/month.
2. Sellego
A similar service focused on price automation. Tracks competitor prices and automatically adjusts your prices according to specified rules (for example, "be 5% cheaper than the competitor"). It also works through its own proxies.
3. Parsehub
A visual no-code scraper builder. You click on the elements of the page you want to collect, and the service automatically creates the scraper. Supports proxies — you can add your list in the project settings. Suitable for those who have no programming skills.
Cost: free plan — 200 pages per month, paid plans — from $149/month for 10,000 pages. For monitoring 100-200 products, the free plan will suffice.
For Self-Development
If you are ready to set up the scraper yourself or have a programmer on your team:
1. Scrapy (Python)
A powerful framework for scraping with built-in support for proxies, rotation, and error handling. There is middleware for automatic proxy switching from a list. Requires knowledge of Python, but the documentation is excellent.
Suitable for scraping simple sites without JavaScript. For Wildberries and Ozon, it needs to be combined with Splash or Selenium for JavaScript processing.
2. Puppeteer / Playwright (JavaScript)
Headless browsers that fully emulate a real user — execute JavaScript, save cookies, and can emulate mouse movements. Ideal for platforms with strict protection.
Proxy setup is simple — one parameter when launching the browser. Playwright is considered more modern and stable, but Puppeteer has more ready-made examples and libraries.
3. Octoparse
A desktop application for Windows with a visual scraper builder. Does not require programming — you simply click on the elements you want to collect. Built-in proxy support, scheduling, export to Excel/CSV.
Cost: free version with limitations (10,000 records per month), paid plans from $75/month. A good option for small and medium businesses.
Step-by-Step Setup of a Price Scraper with Proxies in 15 Minutes
I will show you the easiest way to set up price monitoring using Parsehub — a no-code tool. This method is suitable even if you have never worked with scrapers.
Step 1: Registration and Installation
- Register at parsehub.com (there is a free plan)
- Download and install the desktop application for Windows or Mac
- Launch the application and log into your account
Step 2: Creating a Project and Selecting Elements
- Click "New Project" and paste the URL of the product page (for example, a category on Ozon or search results)
- Parsehub will load the page in the built-in browser
- Click on the name of the first product — Parsehub will automatically highlight all similar elements on the page
- Click on the product price — it will be added to the list of collected data
- Add other necessary fields: product link, rating, number of reviews
Step 3: Setting Up Proxies
- Open the project settings (gear icon)
- Find the "Proxy Settings" section
- Select "Use proxy" and paste your proxy details in the format: IP:PORT:USERNAME:PASSWORD
- If you have a list of proxies, enable the "Rotate proxies" option — Parsehub will automatically change proxies between requests
Step 4: Setting Up a Schedule
- In the project settings, find "Schedule"
- Select the frequency of runs: every hour, every 6 hours, once a day, etc.
- For dynamic pricing, the optimal frequency is every 2-4 hours
Step 5: Exporting Data
- After the first run of the scraper, the data will appear in the "Data" tab
- Click "Export" and choose the format: CSV, Excel, JSON
- You can set up automatic data sending to Google Sheets or via email after each run
The entire setup takes 10-15 minutes. After that, the scraper works automatically on schedule, collects competitor prices, and sends you the data.
Important: For scraping Wildberries and Ozon through Parsehub, be sure to use residential proxies. Data centers will be blocked, even with rotation. A pool of 10-20 proxies is enough to monitor 200-500 products.
5 Mistakes That Lead to Blocking During Scraping
Even with the right proxies, you can get banned if you make typical mistakes. Here are the most common problems from practice.
Mistake 1: Too High Request Speed
Many strive to collect data as quickly as possible and make 10-20 requests per second. This is instantly detected by protection systems. A real person physically cannot open 20 product pages per second.
Solution: Add delays between requests. A minimum of 2-3 seconds for simple sites, 5-10 seconds for marketplaces with protection. Make the delays random — not strictly 5 seconds, but from 4 to 8 seconds randomly.
Mistake 2: Using One Proxy for All Requests
Even if you bought a residential proxy, using one IP for hundreds of requests a day will lead to blocking. Protection systems track the activity of each IP.
Solution: At least 10-20 proxies in the pool with rotation. For large volumes (1000+ products a day) — 50-100 proxies.
Mistake 3: Missing or Incorrect Request Headers
Requests without a User-Agent or with the default User-Agent of the library (for example, "Python-requests/2.28.1") are immediately detected as bots.
Solution: Always specify a User-Agent of a real browser. Additionally, add headers like Accept, Accept-Language, Accept-Encoding. Change the User-Agent when switching proxies.
Mistake 4: Ignoring Cookies and Sessions
Many platforms use cookies to track user sessions. If you make each request "from scratch" without cookies, it looks suspicious.
Solution: Save cookies between requests within one session. When switching proxies — clear cookies and start a new session. Use libraries with automatic cookie management (requests.Session in Python, puppeteer in JavaScript).
Mistake 5: Scraping Only Product Pages Without Simulating Navigation
If you immediately open direct links to product pages, bypassing the main page, search, and categories, it does not resemble the behavior of a real user.
Solution: For platforms with strict protection (Wildberries, Ozon), simulate the full user path: main → search or category → product list → product page. This increases scraping time but reduces the risk of bans significantly.
Conclusion
Monitoring dynamic pricing is a critically important task for any seller on marketplaces. Competitors' prices change several times a day, and those who react first to changes maintain their positions in the search results and profits. However, all major platforms strictly block scraping, so without properly configured proxies, collecting data is impossible.
Key takeaways from the article: for Wildberries and Ozon, use only residential or mobile proxies — data centers are blocked in 90% of cases. Be sure to set up proxy rotation — change IP after every 20-30 requests. Add delays between requests (5-10 seconds) and randomize them. Use headless browsers for platforms with JavaScript protection. Simulate real user behavior — do not open product pages directly, emulate the full path through search and categories.
If you do not want to deal with technical details, use ready-made tools like Parsehub or Octoparse — they do not require programming and have built-in proxy support. For large volumes and specific tasks, it is better to develop your own scraper using Scrapy or Puppeteer.
If you plan to set up automatic price monitoring on Russian marketplaces, we recommend starting with residential proxies — they provide stable access to Wildberries, Ozon, and other platforms with minimal risk of blocks. For particularly challenging cases or working with mobile marketplace applications, mobile proxies with IPs from Russian operators will be suitable.