If you sell on marketplaces or manage an online store, you know that competitor prices change several times a day. Manual monitoring means lost hours and missed profits. Automatic data collection solves the problem, but websites and marketplaces actively block scrapers. In this article, we discuss how to set up price monitoring so that your requests are never detected or blocked.
Why websites block price monitoring and how they detect you
Before setting up monitoring, it is important to understand the logic of protection from the websites. Marketplaces and online stores are well aware that competitors want to track their prices — and they actively counteract this. Protection works on several levels simultaneously.
Analysis of request frequency
A regular buyer visits a product page every few minutes or hours. A scraper makes hundreds of requests per minute from a single IP address. Anti-bot systems (Cloudflare, Imperva, DataDome) instantly notice such an anomaly and block the IP. That’s why working with a single address for automatic data collection is practically impossible.
IP address ownership verification
Each IP address has its own history and ownership. Data center addresses (AWS, Google Cloud, Hetzner) are easily identified by databases — websites know that bots and scrapers operate from such addresses and block them outright. Residential IPs — those belonging to real home users — raise significantly fewer suspicions.
Behavioral analysis
Modern protection systems analyze not only the IP but also behavior: scrolling speed, mouse movements, pauses between actions, the order of resource loading on the page. If requests are made strictly on schedule, without random pauses — this is a signal for blocking.
Geolocation restrictions
Some platforms show different prices depending on the region. Wildberries, for example, may display different prices for Moscow and Novosibirsk. If you are monitoring prices for a specific region, your proxy must have an IP from that region — otherwise, you will receive irrelevant data.
Important to know:
Wildberries and Ozon use several levels of protection simultaneously. Just having a proxy is not enough — proper rotation, correct request headers, and smart configuration of the scraping tool are needed.
Which proxies are suitable for price monitoring: comparison of types
Not all proxies are equally useful for price monitoring. The choice depends on which sites you are monitoring, how often you need data updates, and what budget you are willing to allocate. Let's break down each type in detail.
| Proxy Type | Trust Level | Speed | Best for | Cost |
|---|---|---|---|---|
| Residential | Very High | Average | Wildberries, Ozon, secure sites | Average — High |
| Mobile | Maximum | Average | Platforms with strict protection, mobile versions of sites | High |
| Data Centers | Low | High | Sites without serious protection, small stores | Low |
Residential proxies — the optimal choice for most tasks
Residential proxies use IP addresses of real home internet users. From the website's perspective, it is a regular buyer from an apartment or office accessing it — no suspicions. This makes them ideal for monitoring on Wildberries, Ozon, Avito, and most other platforms. A key advantage is the ability to choose geolocation: you can get an IP from a specific city or region in Russia, which is critical for accurate monitoring of regional prices.
Rotating residential proxies automatically change the IP address with each request or after a specified interval. This means that even if one address gets temporarily restricted, the next request will go from another IP — monitoring does not get interrupted.
Mobile proxies — for the most secure platforms
Mobile IPs are addresses from cellular operators (MTS, Beeline, MegaFon). They have one unique property: behind one mobile IP can stand thousands of real users (through the operator's NAT). Therefore, websites rarely block mobile addresses — the risk of affecting real buyers is too high. If you are monitoring a platform with aggressive anti-bot protection and residential proxies are occasionally blocked — switch to mobile proxies.
Data center proxies — only for simple tasks
Data center proxies work quickly and are cheap, but they are easy to recognize. Most serious marketplaces will block them within minutes of intensive use. They are suitable for monitoring small online stores without serious protection or for preliminary testing of scraping schemes.
Features of data collection on Wildberries, Ozon, Avito, and other platforms
Each major platform has its own protection features. Understanding these features allows you to set up monitoring correctly and not waste resources.
Wildberries
Wildberries is one of the most difficult marketplaces to scrape in Russia. The platform actively uses dynamic content loading via JavaScript, which means: a simple HTTP request will not give you the price — the page must be fully rendered by the browser. Additionally, WB frequently changes the structure of pages and API endpoints, which breaks scrapers.
For Wildberries, rotating residential proxies with Russian IPs are optimal. Request frequency — no more than one request every 3–5 seconds per IP. It is also important to mimic real user behavior: random pauses, correct browser headers (User-Agent, Accept-Language).
Ozon
Ozon uses a protection system based on Cloudflare and its own anti-bot solutions. The platform analyzes the TLS fingerprint (digital fingerprint of the connection) — this means that even with a good proxy, a request may be blocked if it does not look like a request from a real browser. For Ozon, it is recommended to use tools that fully emulate a browser (headless Chrome, Playwright), rather than simple HTTP clients.
Avito
Avito has moderate protection but actively blocks data center IPs. Residential proxies with Russian addresses work reliably. A feature of Avito is regionality: prices and listings vary significantly by city. If you are monitoring competitors in a specific region, be sure to use proxies with IPs from that region; otherwise, the data will be irrelevant.
Yandex.Market and other aggregators
Yandex.Market uses an advanced bot detection system — not surprising given the company's resources. The platform analyzes behavior, request frequency, and connection characteristics. For stable monitoring, rotating residential proxies with pauses between requests of at least 5–10 seconds are needed.
Geolocation advice:
For monitoring Russian marketplaces, always use proxies with Russian IPs. Foreign IPs may yield different prices, different assortments, or may even be blocked at the geolocation level.
Ready-made tools for price monitoring without coding
The good news for those who do not write code: there are ready-made services and tools that allow you to set up competitor price monitoring literally in a few hours. You just need to connect the proxy and specify what exactly to track.
Specialized price monitoring services
Platforms like Priceva, Competera, and similar Russian services provide a ready-made solution: you add the URLs of competitors' pages, set the check schedule, and receive reports in a convenient format. Many of them support connecting your own proxies — this is important if you want to control geolocation and the quality of IP addresses.
No-code scrapers: Octoparse, ParseHub, Apify
Octoparse and ParseHub are visual scrapers with a drag-and-drop interface. You literally "click" on elements of the page (price, product name, availability), and the tool figures out how to extract them. Both support connecting proxies through the standard format host:port:login:password. Apify is a more powerful platform with ready-made "actors" (templates) for scraping popular sites, including marketplaces.
Google Sheets + IMPORTXML/IMPORTHTML
For simple tasks — monitoring 10–20 pages without serious protection — you can use the built-in functions of Google Sheets. IMPORTXML and IMPORTHTML allow you to pull data from web pages directly into the sheet. The downside: it does not support proxies and works only with simple static pages. This method is not suitable for marketplaces.
Anti-detect browsers for manual monitoring
If you need to periodically check prices manually from different regions — for example, to see how your product appears to a buyer from Yekaterinburg — anti-detect browsers like Dolphin Anty, AdsPower, or GoLogin will be a convenient tool. You create a separate browser profile with the desired geolocation, connect a proxy with the IP of the required city, and open the competitor's page. Each profile looks like a separate unique user.
How to set up proxies for monitoring: step-by-step instructions
Let's discuss practical setup using the example of connecting proxies to a popular monitoring tool. The principle is the same for most no-code solutions.
Step 1: Get proxy data
After purchasing a proxy, you receive connection data in the format: host (IP or domain), port, username, and password. For rotating residential proxies, usually one endpoint is provided — the gateway address, through which the IP automatically changes with each request. Write down this data — you will need it in the next step.
Step 2: Choose the protocol — HTTP or SOCKS5
Most scraping tools support both protocols. HTTP proxies are a universal option, working with any tool. SOCKS5 is a more flexible protocol, supporting any type of traffic and better concealing the fact that a proxy is being used. For price monitoring, we recommend SOCKS5 if your tool supports it.
Step 3: Configuration in Octoparse (example)
In Octoparse, go to the menu Settings → Proxy Settings. Select the Custom Proxy mode. Enter your proxy data: host, port, username, password. Click Test Connection — the tool will check the proxy's functionality and show your new IP address. If the test is successful — the proxy is connected. Now all scraper requests will go through it.
Step 4: Set up rotation and delays
This is a critically important step that many skip. Even with a good proxy, aggressive scraping will be detected. Set the following parameters:
- Delay between requests: at least 3–5 seconds for Wildberries and Ozon, 1–2 seconds for less protected sites
- IP rotation: change IP every 10–20 requests or every 5–10 minutes
- Random pauses: instead of a fixed delay, use a random interval (for example, from 3 to 8 seconds)
- User-Agent: use current User-Agent strings from real browsers (Chrome, Firefox)
Step 5: Testing before launch
Before launching full-scale monitoring, conduct a test: run scraping on 10–20 pages and check if all data is collected correctly and if there are any blocks. If some requests return a 403 error (access denied) or CAPTCHA — increase the delays and check the rotation settings.
Common mistakes when scraping prices and how to avoid them
Most problems with price monitoring arise from the same mistakes. Let's discuss the most common ones.
Mistake 1: Using a single IP for all requests
Even if you have a good residential IP — do not use it for hundreds of requests in a row. Any IP making too many requests in a short time will be blocked. Use rotating proxies or a pool of several IP addresses, distributing the load among them.
Mistake 2: Ignoring geolocation
If you are monitoring competitor prices in Moscow using proxies with IPs from Novosibirsk or even from Germany — you will get incorrect data. On Wildberries, Ozon, and Avito, prices and delivery conditions depend on the region. Always choose proxies with geolocation corresponding to your task.
Mistake 3: Too high request frequency
The desire to get data as quickly as possible leads to the scraper making requests every 0.1–0.5 seconds. Such activity is instantly detected. Prices do not change every second — usually, it is sufficient to check them once an hour or even once every few hours. Reduce the frequency — and the stability of monitoring will increase sharply.
Mistake 4: Cheap data center proxies for complex platforms
Many start with the cheapest data center proxies and are surprised why Wildberries blocks them in 5 minutes. Serious marketplaces require residential or mobile proxies. Saving on proxies results in wasted time fighting blocks.
Mistake 5: Lack of monitoring proxy performance
Proxies may temporarily not work, have high latency, or return errors. If you do not monitor this, the monitoring can silently "break" — data will stop updating, and you will only find out about it after several days. Set up alerts for errors in your tool or periodically check the relevance of the data manually.
Working monitoring schemes for different tasks
To avoid reinventing the wheel, let's discuss specific schemes for typical tasks of sellers and marketers.
Scheme 1: Monitoring competitor prices on Wildberries — for sellers
Task: Track prices of 50–100 competing products on Wildberries, update every 2–4 hours.
Tools: Specialized service (Priceva or similar) or Apify with a ready-made actor for WB.
Proxies: Rotating residential proxies with Russian IPs. A pool of at least 50 addresses to distribute the load.
Settings: Delay of 5–8 seconds between requests, random interval, IP rotation every 15 requests.
Result: Up-to-date data on competitor prices every 2–4 hours, automatic alerts for price changes of more than 5%.
Scheme 2: Regional price monitoring on Avito — for local businesses
Task: Monitor competitor prices on Avito in a specific city (e.g., Krasnodar), update once a day.
Tools: Octoparse or ParseHub with a configured template for Avito.
Proxies: Residential proxies with IPs from the specific city or region. Important: the IP must correspond to the desired city to obtain local prices.
Settings: Run once a day, delay of 3–5 seconds, static IP (aggressive rotation is not needed with low frequency).
Result: Daily report on competitor prices in your region, export to Excel or Google Sheets.
Scheme 3: Monitoring competitors' advertising offers — for marketers
Task: Track promotions, discounts, and special offers from competitors on their websites and in advertisements.
Tools: A combination of a scraper (Apify) for websites + advertising monitoring tools for tracking ads.
Proxies: Residential proxies with geolocation of the target market. If monitoring ads for different regions — proxies from several regions are needed simultaneously.
Settings: Check 2–3 times a day, IP rotation with each new session.
Result: Timely information on competitors' marketing activities, ability to quickly respond to their promotions.
Scheme 4: Monitoring prices on foreign marketplaces — for importers
Task: Track prices from suppliers and competitors on AliExpress, Amazon, eBay.
Tools: Apify with ready-made actors for Amazon and AliExpress, or specialized monitoring services.
Proxies: Residential proxies with IPs from the country of the target marketplace (USA for Amazon, China or USA for AliExpress). This is important, as prices may vary depending on the visitor's country.
Settings: Update every 6–12 hours, delay of 5–10 seconds, IP rotation every 20 requests.
Checklist before launching price monitoring:
- ✅ Proxy type selected for the task (residential for marketplaces)
- ✅ Proxy geolocation matches the target market
- ✅ IP rotation configured
- ✅ Delays between requests set (minimum 3–5 seconds)
- ✅ Tested on 10–20 pages before full launch
- ✅ Alerts set up for errors or price changes
- ✅ Update schedule defined (not more frequently than necessary)
Conclusion
Automatic monitoring of competitor prices is not just a convenience, but a competitive advantage. Knowing how prices change on Wildberries, Ozon, Avito, and other platforms allows you to respond quickly: lowering prices when competitors are undercutting, raising them during periods of high demand, launching promotions at the right moment. All of this directly impacts sales and profitability.
The key to stable monitoring is properly selected proxies and smart configuration of tools. For most Russian marketplaces, the optimal choice is rotating residential proxies with Russian IPs: they appear as real users, support geolocation selection, and are not blocked by aggressive anti-bot systems.
If you are working with platforms that particularly harshly block scrapers, or if you need maximum stability — consider mobile proxies: they are rarely blocked since thousands of real users can stand behind one mobile IP.
Start small: set up monitoring for 20–30 key competitor positions, test the scheme, ensure data stability — and then scale up. Properly configured price monitoring pays off within the first month of operation.