Your Position Tracker Instead of Topvisor and Serpstat: Cost Calculation for 1000 Keywords per Day

SEO agencies and in-house marketers eventually ask themselves the same question: why pay for a subscription to a rank tracker when you can build your own scraper? The answer is not as simple as it seems — the cost of self-monitoring consists not only of servers but also of proxies, captcha, and the time spent maintaining scripts. In this article, we will break down the real math using the example of checking 1000 keywords per day.

Why agencies create their own rank tracker

Ready-made services like Topvisor, Serpstat, SE Ranking, or Rush Analytics are convenient: you don’t have to think about proxies, captcha, and servers — just enter the keywords and get a report. But this convenience comes at a price, which grows proportionally to the number of checked phrases and regions. If an agency manages 15-20 clients, each with 500-2000 keywords, the subscription bill quickly reaches an amount comparable to the salary of a junior specialist.

The second reason is flexibility. In a ready-made service, you cannot change the checking frequency for individual keywords, add custom logic (for example, track not only the position but also the snippet, the presence of quick links, or reviews), or export raw data to your BI system. A custom tracker solves these tasks but requires infrastructure: a server, a database, parsing scripts, and, most importantly, proxies to bypass search engines' protection against automated requests.

The third reason is control over data. Agencies that work with large clients under NDA are not always willing to share the list of keywords and domains with third-party cloud services. An in-house tracker allows storing everything on their own infrastructure.

What constitutes the cost of position checking

Before calculating costs, it’s important to understand the mechanics: to find out the position of a site for a keyword, the scraper must send a request to Yandex or Google’s results page, receive the HTML results page, and find the required domain within it. For 1000 keywords per day, this means at least 1000 requests, and if you check both search engines and several regions, the number multiplies by 2-4 times.

Search engines aggressively block automated requests from a single IP: after 20-50 consecutive requests from one address, Yandex starts showing captcha, while Google temporarily bans the IP for several hours. Therefore, each request must go through a separate or frequently changing proxy. This leads to the main expense item — not the server or the script, but the proxy infrastructure.

The complete cost structure looks like this:

• Proxies for IP rotation — the main variable expense item, depends on the volume of requests
• Server or VPS for running scripts on schedule — a small fixed amount
• Captcha solving (anti-captcha services) — an additional expense if the proxies are of insufficient quality
• Developer time for writing and maintaining the scraper — a one-time and periodic cost
• Data storage and visualization — database, dashboard, usually minimal expenses

Subscription vs own tracker: comparison of expense items

To assess when an own tracker is more profitable than a subscription, it is convenient to compare expense items directly. Below is a summarized table not tied to specific tariffs, showing the logic of choice.

Parameter	Ready-made service (subscription)	Own rank tracker
Cost for 1000 keywords/day	Grows proportionally to the tariff plan, fixed monthly amount	Consists of proxy traffic + server rental, scales smoothly
Flexibility of settings	Limited by the service's functionality	Complete — any frequency, region, depth of results
Developer needed	No	Yes, at the start and periodically for support
Risk of bans/captcha	Assumed by the service	Falls on you, resolved by proxies and delays
Break-even point	—	Usually from 3000-5000 keywords per day and above

The conclusion from the table is simple: with a small volume of keywords (up to 2000-3000 per day), a subscription is almost always more profitable because the costs of developing and maintaining your own solution do not pay off. But with larger volumes — from several thousand keywords per day, especially if you need to check them several times a day or in different regions — your own infrastructure starts to win in terms of economics.

Proxy calculation for 1000 keywords per day

The main question regarding cost is how many proxies are needed for 1000 requests per day without bans and captcha. Let’s take the average safe limits that scrapers typically use:

• Yandex: 30-50 requests from one IP per day without the risk of captcha with random delays between requests of 10-30 seconds
• Google: 15-25 requests from one IP per day, the system bans more aggressively and quickly

If you are checking 1000 keywords only in Yandex, with a limit of 40 requests per IP, you need at least 25 unique addresses per day. Considering retries for unsuccessful requests and a buffer for temporary bans, plan for 30-40 IPs. If you check both search engines, the number doubles — you will need about 60-80 IPs per day.

An important nuance: when working through a pool of IPs with rotation per request (rather than per session), the same physical proxy server can "cover" many more IP addresses if the provider offers dynamic rotation. In this case, the calculation is based not on the number of addresses but on the volume of traffic — usually, one position check weighs 50-200 KB of HTML, which means 1000 requests per day result in 50-200 MB of traffic, which is not critical even for an average tariff.

What proxies are needed for Yandex and Google

The type of proxy directly affects how much captcha and bans you will encounter, and thus the actual cost of checking. Let’s break down three options.

Data center proxies — the cheapest option in terms of traffic cost, but they have a bad reputation in search engine databases: data center IPs are widely used for scraping, so Yandex and Google quickly recognize them and show captcha after just a few requests. For low-frequency checks of a small number of keywords, data center proxies can still be used, but at a volume of 1000+ requests per day, the share of captcha becomes too high, and savings on traffic are eaten up by anti-captcha expenses.

Residential proxies — IP addresses of real home users, which search engines perceive as regular traffic. The share of captcha here is significantly lower, which is critical for large volumes of checking. For scraping results for 1000 or more keywords per day, residential proxies usually provide the best cost-to-stability ratio, especially when working with Google, which bans data center addresses the hardest.

Mobile proxies — use IPs from mobile operators, which often change among thousands of real subscribers (NAT), so the same address can "belong" to different users throughout the day. This provides maximum longevity during aggressive scraping but is more expensive. Mobile proxies are justified if you are checking positions in mobile results separately from desktop ones or working with particularly capricious regional queries from Yandex.

Proxy Type	Share of captcha per 1000 requests	When to use
Data center	High	Up to 200-300 keywords per day, low frequency
Residential	Low	1000+ keywords per day, Google and Yandex
Mobile	Minimal	Mobile results, complex regions

Step-by-step setup of your tracker

Let’s break down the process of building your rank tracker step by step — without deep diving into the code, at the architectural level of the solution.

1. Choosing a data source. Determine whether you are scraping the search engine results directly via HTTP requests or using a headless browser (e.g., through Puppeteer/Playwright) to simulate a real user. The second option is more reliable against anti-bot protection but slower and more resource-intensive on the server.
2. Setting up a proxy pool. Connect to a proxy provider with IP rotation per request or time. For Yandex and Google, it’s better to use different proxy pools, as the systems have different blocking thresholds.
3. Retry logic. If a request returns a captcha or error, the script should automatically change the IP and repeat the request, rather than just logging the error.
4. Parsing the result. The position of the required domain in the results is extracted from the received HTML — usually by searching through link tags and comparing with the client’s domain.
5. Storing history. Results are saved in a database with the date of the check, allowing for the construction of position dynamics graphs.
6. Scheduling launches. The script is run via cron or task scheduler once a day (or more frequently), evenly distributing 1000 requests over time to avoid creating peak loads on the proxy pool.

Example code for checking positions in Python

Below is a simplified example of the logic for making a request to the search results through rotating proxies. This is a demonstration framework that shows the principle of operation, not a ready-made solution.

import requests
import random
import time

PROXY_LIST = [
    "http://user:[email protected]:8000",
    "http://user:[email protected]:8000",
    # ... pool of proxies from hundreds of addresses
]

HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
}

def check_position(keyword, domain, search_engine="yandex"):
    proxy = random.choice(PROXY_LIST)
    proxies = {"http": proxy, "https": proxy}

    if search_engine == "yandex":
        url = f"https://yandex.ru/search/?text={keyword}"
    else:
        url = f"https://www.google.com/search?q={keyword}"

    try:
        response = requests.get(url, headers=HEADERS, proxies=proxies, timeout=10)
        if "captcha" in response.text.lower():
            return None  # IP rotation and request retry needed
        position = find_domain_position(response.text, domain)
        return position
    except requests.RequestException:
        return None

def find_domain_position(html, domain):
    # simplified logic for finding domain position in results
    results = extract_links(html)
    for i, link in enumerate(results, start=1):
        if domain in link:
            return i
    return None

for keyword in keyword_list:
    pos = check_position(keyword, "example.com")
    save_to_db(keyword, pos)
    time.sleep(random.uniform(5, 15))  # random delay between requests
  

In a real project, instead of manual HTML parsing, a headless browser is often used with User-Agent and fingerprint spoofing, and the proxy pool is connected through a provider with automatic rotation at the endpoint level — this removes the need to manually maintain a list of addresses and simplifies scaling to several thousand requests per day.

How much can you really save

To answer the question of savings, you need to compare two models of expenses dynamically as the volume of keywords grows.

With a small volume — up to 1000-2000 keywords per day — the cost of a subscription to a ready-made tracker is usually comparable to or even lower than the total costs of developing and maintaining your own solution, considering the time of the specialist. Developing a minimally viable scraper from scratch takes from several days to a couple of weeks, plus periodic support when the search results markup changes — and Yandex and Google often change the layout of results.

The turning point occurs when the volume of checks grows to several thousand keywords per day or when positions need to be checked several times a day, in different regions and devices (desktop/mobile). In this scenario, the cost of a ready-made subscription grows linearly or even progressively with volume, while the expenses for your own tracker grow much more slowly — the main item (proxy traffic) does not increase as sharply as the tariff plans of SaaS services when moving to the next package level.

A practical guideline for agencies: if you are tracking positions for 5-10 or more clients simultaneously, the total volume usually exceeds 5000-10000 keywords per day — it is at this scale that your own tracker is almost always more profitable than any ready-made subscription, provided that there is someone on staff capable of maintaining the scraper.

Common mistakes in self-scraping

In practice, most problems with your rank tracker are related not to the code but to underestimating infrastructure nuances.

• Saving on proxies. Trying to save on cheap data center proxies leads to mass captchas that need to be solved through third-party anti-captcha services — in the end, total expenses turn out to be higher than if higher-quality proxies had been used from the start.
• Lack of delays between requests. Too fast consecutive requests from one IP instantly attract the attention of anti-bot systems, even if the IP is of high quality.
• Ignoring regionality. Yandex shows different results depending on the user's region — without proper geo-targeting of proxies, data on positions will be incorrect.
• Lack of monitoring the health of the scraper. Search engines periodically change the layout of the results page, and without alerts for drops in parsing success, you can collect empty or incorrect data for weeks without noticing it.
• Ignoring mobile results. Desktop and mobile results often differ, and if the mobile position is important to the client, a separate scraping contour with appropriate User-Agent and, preferably, mobile IPs is needed.

Conclusion

An own rank tracker becomes economically justified when the volume of checks exceeds several thousand keywords per day, and there is a resource in the team to support the scraper. At smaller volumes, a ready-made subscription is almost always cheaper considering hidden development costs. The main variable in the cost of your solution is not the server or the code, but the proxy infrastructure, which determines the share of captcha and the stability of the data.

If you plan to build a rank tracker for 1000 or more keywords per day, start with calculating traffic and testing a proxy pool on a small sample of queries. For stable scraping of Yandex and Google at such volumes, it is reasonable to immediately consider residential proxies — they provide a minimal share of captcha at an acceptable traffic cost and scale well with the growth of the number of clients being tracked.