Accept-Language, currency, and geo-cookie: fixing the parser.

The parser runs, the code doesn't crash, data comes in — but prices are in dollars instead of rubles, the text is in English, and the products differ from those seen by a live user in Russia. This is one of the most common and frustrating problems in parsing: technically everything works, but the content is "not right." We analyze why this happens and how to construct a request that retrieves exactly the version of the page you need.

Symptoms: how to understand that the parser sees the wrong version

Before fixing the problem, it is important to diagnose it correctly. There are three typical scenarios in which the parser receives data different from what a regular browser user sees:

The content language does not match — you expect Russian text, but receive the English or Ukrainian version of the page.
The currency and prices differ — the site shows prices in USD or EUR, while in the browser from Russia you see rubles, or the numbers are simply different (regional pricing).
The product/content catalog is different — the availability of products, promotions, and the entire storefront depend on the region that the site "saw" from you.

The reason is almost always one of three (or a combination of them): the Accept-Language header, missing or incorrect cookies, or the geolocation of the IP address from which the request is made. Let's break down each in order.

Accept-Language: how the server chooses the response language

The browser sends the Accept-Language header with each request, which lists the user's preferred languages with weights (q-values). Here is an example of what Chrome sends with a Russian locale:

Accept-Language: ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7

Parsing libraries (requests, httpx, scrapy) by default either do not send this header at all, or send something like */*. In this case, the server either returns the default language (usually English), or determines the language by IP — and this is where the geo part comes into play, which we will discuss below.

An important nuance: many sites use a combination of "Accept-Language + IP geolocation," rather than just one. This means that even if you send the correct language header, if the request comes from an IP of another country, the site may still switch you to the English or local version for the region from which the request actually came.

Currency and locale: where the site gets its prices

Currency is almost never determined directly by the Accept-Language header. More often, one of three mechanisms (or their combination) is used:

GeoIP database — the server checks from which country the IP came and automatically assigns the corresponding currency and locale on the first visit.
Cookie with the previously selected region — if the user manually selected a country and currency at some point, the site saves this in a cookie and no longer checks the IP.
Parameter in the URL or subdomain — some platforms use explicit markers like ?country=ru or ru.example.com.

In practice, if you are parsing an online store and see prices in dollars instead of rubles, in 80% of cases the issue is not with the headers, but specifically with the IP address of the request. The site simply "saw" a non-Russian IP and displayed the default international storefront.

After the site has determined your region once (either by IP or by manual selection on the "Choose your city" banner), it saves this decision in a cookie — usually with names like region, geo_city, currency, or locale. All subsequent requests with this cookie are processed according to the saved region — even if the IP suddenly changes.

This creates a trap for the parser: if you once received the "wrong" version and saved the session cookie, all subsequent requests will reproduce the same error, even if you fix the headers or proxies. Therefore, when diagnosing, always check parsing with a clean session — without saved cookies from previous runs.

The flip side of the same mechanism is useful: if you know the name of the required cookie and its correct value for the target region (for example, you spotted it in the browser's DevTools in the desired location), you can forcibly set this value in the request without changing the IP. This works for sites where the region is determined only through a cookie flag, but does not work if the server checks the IP against the GeoIP database with each request.

The role of IP address and proxies in determining the region

GeoIP determination is the most reliable way for a site to understand where the request actually came from, because headers and cookies can be faked, but an IP address cannot be faked (it can only be replaced with another real one). This is why, for stable retrieval of the "correct" regional version of the page, it is not enough to rely solely on HTTP headers — you need an IP that truly belongs to the desired country.

There is a significant difference between types of IP addresses:

Data center IPs — GeoIP databases often correctly identify them by country, but many large sites (especially marketplaces and services with bot protection) mark data center ranges as suspicious and may show a limited or default version of the content regardless of geolocation.
Residential IPs — belong to real providers and physical addresses, so GeoIP determines the region more accurately, and the site perceives the request as coming from an ordinary user in that country/city.
Mobile IPs — tied to mobile operators of a specific country, which is especially important for sites and applications that handle mobile and desktop traffic differently.

For tasks like monitoring prices on Wildberries, Ozon, or Avito from specific cities, the best fit is residential proxies — they provide IPs tied to real regions, and GeoIP databases of sites correctly identify the country and city without switching to the default international storefront.

Practice: constructing the correct request in Python

Let's construct a request that simultaneously solves all three problems: sends the correct Accept-Language, uses a clean session with the necessary cookies, and goes through the IP of the required region.

import requests

proxies = {
    "http": "http://user:[email protected]:8080",
    "https": "http://user:[email protected]:8080",
}

headers = {
    "Accept-Language": "ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                  "AppleWebKit/537.36 (KHTML, like Gecko) "
                  "Chrome/124.0.0.0 Safari/537.36",
}

cookies = {
    "region": "ru",
    "currency": "RUB",
}

session = requests.Session()
response = session.get(
    "https://example.com/catalog",
    headers=headers,
    cookies=cookies,
    proxies=proxies,
    timeout=15,
)

print(response.status_code)
print(response.text[:500])

Several important points in this code:

Using requests.Session(), rather than a single requests.get(), allows the server to set additional cookies if necessary, and they are saved between requests of the same parsing session.
The values of the cookies region and currency are conditional, their actual names and values need to be checked in the browser's DevTools (Application tab → Cookies) on the target site with the desired location.
The proxy must actually be located in the desired country — otherwise, even correct headers and cookies may be overwritten by the server when rechecking geolocation.

JS sites: Playwright and managing headers/cookies

If the site renders content through JavaScript (prices are loaded via a separate request after the page loads), a simple requests request is not enough — a headless browser is needed. In Playwright, the locale, language, and proxy are set at the browser context level:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    context = browser.new_context(
        locale="ru-RU",
        timezone_id="Europe/Moscow",
        proxy={
            "server": "http://gate.proxycove.com:8080",
            "username": "user",
            "password": "pass",
        },
    )
    context.add_cookies([
        {"name": "region", "value": "ru", "domain": "example.com", "path": "/"},
        {"name": "currency", "value": "RUB", "domain": "example.com", "path": "/"},
    ])

    page = context.new_page()
    page.goto("https://example.com/catalog")
    page.wait_for_selector(".price")
    print(page.content()[:500])

    browser.close()

Note the timezone_id parameter — many sites additionally check the browser's time zone against the expected one for the region, and a discrepancy (for example, IP from Russia, but the browser's time zone is American) can itself be a signal for anti-bot systems that the request is suspicious, even if the language and currency are formally correct.

Common mistakes and a checklist before launching

Before launching the parser in production, go through a short checklist — this saves hours of debugging:

✅ The Accept-Language header is set and matches the expected region.
✅ The session is clean — cookies are not inherited from a previous unsuccessful request.
✅ The proxy's IP is physically located in the desired country (check with any geo-IP checker before parsing).
✅ The User-Agent matches a real browser and does not contain signs of headless mode.
✅ The time zone and locale of the browser (for Playwright/Selenium) match the country of the IP.
✅ If the site uses manual region selection on a banner — the cookie of that selection is passed manually.

It is worth mentioning separately about proxy rotation: if you change the IP for each request, while the session cookies remain the same, the site may see a mismatch of "region A cookie, but region B IP" and reset the session. For price monitoring tasks, it is better to stick to one IP for the entire parsing session of a specific region, rather than rotating it in the middle of data collection.

What type of proxy to choose for the task

The choice of proxy type directly affects how accurately the site will determine your region and not suspect automation. Here is a comparison for typical tasks of parsing regional versions of sites:

Proxy Type	Geolocation Accuracy	When to Use
Data Center Proxies	Average, country is correct, but IP may be marked as "non-residential"	Parsing sites without strict anti-bot protection, high speed and volume of requests
Residential Proxies	High, down to the city level	Price monitoring on marketplaces, regional content, bypassing GeoIP checks
Mobile Proxies	High, plus mobile operator tagging	Sites and applications that handle mobile traffic differently in specific countries

If the task is specifically monitoring the catalog and prices on marketplaces from different cities, residential proxies usually provide the most predictable result: GeoIP databases see a real residential address, not a data center range, which some sites process with separate logic.

Conclusion

The problem of "the parser sees the wrong version of the page" almost always consists of three factors: the Accept-Language header, which determines the preferred response language; cookies that remember the previously selected region and currency; and the geolocation of the IP address, which the server uses to determine the country of the request on the first visit. To get exactly the version of the site that a live user sees, you need to synchronize all three parameters simultaneously, rather than fixing them separately.

If you regularly parse regional versions of sites, monitor competitors' prices on marketplaces, or collect data from different countries, it is worth starting with the choice of the appropriate IP — the other settings (headers, cookies, time zone) are built around it. For accurate targeting of the desired region, we recommend trying residential proxies, which provide predictable geolocation and minimize the risk of getting the default international version of the site instead of the local one.