If you are collecting rental prices for apartments in several cities at once—for market analysis, pricing, or reselling data to realtors—you will eventually run into CAPTCHAs, temporary blocks, and unpredictable traffic consumption. In this article, we will discuss a specific scheme for collecting listings from Avito, Cian, and Yandex.Real Estate across 5 cities, and calculate how many gigabytes of traffic on proxies you actually need to budget for.
Why Monitor Rental Prices in Several Cities
Monitoring rental prices across several cities is necessary not only for real estate agencies. It is a task for market analysts, property management companies, real estate investors, and services that sell rental market reports. Comparing prices in Moscow, St. Petersburg, Kazan, Yekaterinburg, and Novosibirsk allows you to see demand dynamics, seasonality, and anomalies—for example, a sharp drop in prices in an area due to a mass listing of apartments by a single owner.
For real estate agencies, it is also a way to quickly respond to competitor underpricing: if a listing appears in a neighboring residential complex that is 15% cheaper than the market, it needs to be seen within an hour, not a week later. Manually reviewing listings through a browser for 5 cities takes 3-4 hours a day for one employee—automation with proxies reduces this time to 10-15 minutes for checking results.
An important nuance: listing platforms (Avito, Cian, Domclick) actively protect against automated data collection precisely because such reports compete with their own paid analytics services. Therefore, any monitoring scheme is built around three things: simulating the behavior of a live user, diversity of IP addresses, and controlling request speed.
Which Sources of Listings to Use
For a complete picture of the rental market in 5 cities, it is usually necessary to use 2-3 sources simultaneously, as no single platform covers the market completely:
- Avito Real Estate — the largest coverage of private listings, but the strictest protection against parsing and frequent CAPTCHAs with suspicious activity.
- Cian — better represented in large cities, many listings from agencies, and the page structure is more stable for collection.
- Yandex.Real Estate — an aggregator that pulls listings from other platforms, useful for cross-checking prices.
- Domclick — relevant for million-plus cities, fewer blocks, but also fewer listings from individuals.
To start, Avito and Cian are usually sufficient—they provide 70-80% coverage of the rental market in million-plus cities. Other sources are connected when deeper analytics or checking of anomalous prices is needed.
Collection Scheme: Proxies, Tools, and Frequency
The basic scheme for monitoring rental prices in 5 cities is built on the following logic: a separate pool of IP addresses is allocated for each city and each platform, requests are made at intervals that simulate human behavior, and data collection occurs on a schedule several times a day.
In practice, the scheme looks like this:
- A ready-made solution for parsing listings (without writing code) is configured to collect a list of listings based on filters: city, type of housing, price range.
- For each city, a separate profile is created linked to geo-proxies of the corresponding region—this reduces the likelihood of CAPTCHAs, as the IP matches the real location of the sample.
- Requests are distributed over time: not in a continuous stream, but with pauses of 3-8 seconds between listing cards.
- Data collection for each city is repeated 2-4 times a day—this is enough to track new listings and price changes without creating an abnormal load on the platform.
- Results are saved in a table or database linked to the date and time of collection for building price dynamics graphs.
To manage multiple collection profiles by city, it is convenient to use anti-detect browsers like Dolphin Anty, AdsPower, or Octo Browser—they allow you to assign a separate browser profile with unique cookies, user-agent, and, most importantly, the necessary regional proxy to each city. This is the same principle that arbitrageurs use for multi-accounting, only applied to the task of parsing listings.
Traffic Consumption Calculation for 5 Cities
Traffic consumption is a key parameter when planning the budget for proxies, especially if you are using residential IPs, where payment is based on the volume of data transmitted. Let's consider the calculation using the example of monitoring five million-plus cities: Moscow, St. Petersburg, Kazan, Yekaterinburg, Novosibirsk.
Basic inputs for the calculation:
- The average weight of a page with a list of listings (30-50 cards) is 800 KB - 1.5 MB, considering images, if images are not blocked, and about 150-250 KB without loading images.
- The average weight of a single listing card during detailed parsing is 300-500 KB with photos, 40-70 KB in text mode.
- The number of listings to collect per city in one pass is 300-800 depending on the market size (Moscow and St. Petersburg—more, Kazan, Yekaterinburg, Novosibirsk—less).
| City | Listings/Pass | Traffic Without Photos/Pass | Traffic/Day (3 Passes) |
|---|---|---|---|
| Moscow | 800 | ~45 MB | ~135 MB |
| St. Petersburg | 650 | ~37 MB | ~111 MB |
| Kazan | 350 | ~20 MB | ~60 MB |
| Yekaterinburg | 380 | ~22 MB | ~66 MB |
| Novosibirsk | 400 | ~23 MB | ~69 MB |
In total, we get about 440 MB of traffic per day for 5 cities with three passes a day and image loading turned off. In a month, this is approximately 13 GB. If you also collect photos of listings for visual verification (for example, to distinguish real apartments from fraudulent listings), the consumption increases 6-8 times—to 80-100 GB per month for 5 cities.
Practical advice: initially disable image and style loading at the level of the parser or browser profile settings—this reduces traffic significantly without losing data on price, address, and area of the apartment. Photos can be selectively loaded only for listings with suspiciously low prices.
Which Proxies to Choose for Each Source
The choice of proxy type directly affects how many CAPTCHAs you will encounter and how stable the data collection will be. For monitoring listings across 5 cities, different platforms require different approaches.
| Platform | Recommended Proxy Type | Reason |
|---|---|---|
| Avito | Residential Proxies | Strict protection against data center IPs, real provider addresses are needed |
| Cian | Residential Proxies | Tracks request patterns, geo-linking to the city is required |
| Yandex.Real Estate | Data Center Proxies | Protection is weaker, can save on speed and cost |
| Domclick | Data Center Proxies | Moderate protection, high collection speed is more important than anonymity |
For Avito, it is worth mentioning mobile proxies separately—if you are collecting data through a mobile application or emulating mobile traffic, the platform less frequently marks such requests as suspicious since mobile IPs from telecom operators are constantly reused by different real users. This is relevant for real estate monitoring tasks if desktop residential IPs frequently encounter CAPTCHAs in a specific region.
IP Rotation and Request Frequency Without Bans
A common mistake when monitoring rental prices is trying to collect data for all 5 cities from the same IP address sequentially. Platforms see a sharp spike in requests from one IP to different geo-sections of the site and consider it automation, even when adhering to pauses between requests.
A working rotation scheme looks like this:
- Assign a separate IP (or a small pool of 3-5 IPs) to each city for the entire collection cycle—this makes sense, as a real user from Kazan cannot physically view listings in Novosibirsk at the same time.
- Change IPs within the pool every 30-50 listing card requests, rather than on every request—too frequent IP changes also look suspicious.
- Make pauses of 4-10 seconds between listing card requests and 1-3 minutes between passes through different sections of the site.
- Start collection for 5 cities not strictly in parallel, but with a slight time offset—5-15 minutes between the start of collection for each city.
With this approach, a pool of 15-25 residential IPs with session rotation is usually sufficient for 5 cities if the collection volume does not exceed 500-800 listings per city per pass. For more aggressive collection (for example, hourly monitoring), the pool should be increased to 40-50 IPs to keep the load on each address within limits similar to the behavior of an ordinary user.
Common Mistakes When Monitoring Listings
Based on typical cases, several mistakes can be identified that most often lead to blocks and disruptions in data collection across several cities:
- One IP for All Cities — the platform sees an illogical geolocation and restricts access faster than when using local IPs for each region.
- Too Frequent Collection — monitoring every 10-15 minutes is rarely justified for the rental market; prices do not change that quickly, and the load on the source increases proportionally.
- Ignoring User-Agent and Browser Headers — a mismatch between the mobile operator's IP and the desktop user-agent immediately increases the risk of blocking.
- Lack of a Backup Proxy Pool — if all traffic goes through one type of proxy without a backup pool, a mass block halts data collection completely for several hours.
- Collection Without Limiting Photos — leads to traffic overconsumption by 5-8 times without additional value for price analytics.
These mistakes can be easily eliminated at the setup stage—it's enough to pre-divide the proxy pool by cities, limit the frequency of requests, and disable the loading of unnecessary content.
Conclusion and Recommendations
Monitoring rental prices through listings in 5 cities is a task that can be solved without writing code if the scheme is correctly structured: a separate geo-proxy for each city, reasonable request frequency, disabling unnecessary media files, and IP rotation within the assigned pool. With this approach, traffic consumption fits within 10-15 GB per month without photos and 80-100 GB with full image loading of listings.
To start data collection on Avito and Cian, we recommend using residential proxies—they have the lowest CAPTCHA rate when working with geo-sensitive real estate platforms. If the budget is limited or the source has weak protection (for example, aggregators like Yandex.Real Estate), you can switch to cheaper data center proxies without losing data collection quality.