Proxy Integration with REST API Clients: Setting Up Postman, Insomnia, cURL, and HTTPie

When developing and testing APIs, there is often a need to send requests through proxy servers. This may be necessary to bypass geographical restrictions, test API behavior from different regions, ensure anonymity, or work with services that block data center IPs. In this guide, we will discuss how to properly configure proxies in popular REST API clients and avoid common mistakes.

We will cover four of the most popular tools for working with APIs: Postman (the graphical interface for most developers), Insomnia (a modern alternative with a user-friendly UI), cURL (the classic command-line tool), and HTTPie (a modern CLI client with human-readable syntax). For each tool, we will provide code examples and discuss configuration specifics.

What types of proxies are suitable for working with APIs

Before configuring proxies in API clients, it is important to understand which type of proxy is suitable for your task. REST API clients support two main protocols: HTTP/HTTPS and SOCKS5. The choice depends on the specific API requirements and the level of security you need.

HTTP/HTTPS proxies operate at the HTTP protocol level and are ideal for most REST API requests. They understand the structure of HTTP traffic, can cache responses, and modify headers. HTTPS proxies additionally encrypt the connection between the client and the proxy server, which is important when transmitting sensitive data such as API keys or authorization tokens.

SOCKS5 proxies operate at a lower level and transmit any type of traffic without modification. They are slower than HTTP proxies but provide better anonymity and are suitable for cases where the API uses non-standard protocols or requires full connection transparency. SOCKS5 also supports UDP traffic, although this is rarely needed for REST APIs.

Configuring proxies in Postman: global and local settings

Postman is the most popular graphical tool for working with REST APIs, used by millions of developers. It supports two ways to configure proxies: global settings (applied to all requests) and environment-level settings. Let's examine both options in detail.

Global proxy settings in Postman

Global proxy settings are located in the Settings menu (gear icon in the upper right corner). This method is convenient when you want all requests from Postman to always go through the same proxy server. Step-by-step instructions:

1. Open Postman and click on the gear icon (Settings) in the upper right corner
2. Go to the "Proxy" tab
3. Enable the "Add a custom proxy configuration" option
4. Select the proxy type: HTTP, HTTPS, or SOCKS5
5. Enter the proxy server address (e.g., proxy.example.com)
6. Specify the port (usually 8080 for HTTP, 1080 for SOCKS5)
7. If the proxy requires authentication, enable "Proxy Auth" and enter your username/password
8. Click "Save" and close the settings window

After this, all requests from Postman will automatically go through the specified proxy. An important feature: Postman also sends telemetry and checks for updates through the proxy, which can create additional traffic. If you are using a proxy with limited traffic, this should be taken into account.

Configuring proxies via environment variables

A more flexible way is to use environment variables. This allows you to quickly switch between different proxies for different projects or APIs. Create a new environment and add the following variables:

PROXY_HOST = proxy.example.com
PROXY_PORT = 8080
PROXY_USER = your_username
PROXY_PASS = your_password

Then in the global proxy settings, use these variables instead of specific values: {{PROXY_HOST}} and {{PROXY_PORT}}. Now you can create multiple environments (e.g., "Production Proxy", "Testing Proxy", "US Proxy") and switch between them with one click.

Using system proxy

Postman can also use the system proxy settings of your operating system. To do this, in the Proxy settings, select the "Use System Proxy" option. This is convenient if you have already configured the proxy at the OS level (e.g., through Windows or macOS settings) and want Postman to use the same settings as your browser.

Configuring proxies in Insomnia

Insomnia is a modern alternative to Postman with a clean interface and open source. Configuring proxies in Insomnia is slightly different from Postman and requires editing the configuration file or using environment variables. Let's explore both methods.

Configuring via environment variables

The simplest way to configure a proxy in Insomnia is to use the standard HTTP_PROXY and HTTPS_PROXY environment variables. Insomnia automatically reads these variables from the system. Set them before launching Insomnia:

Linux/macOS:

export HTTP_PROXY="http://username:password@proxy.example.com:8080"
export HTTPS_PROXY="http://username:password@proxy.example.com:8080"
insomnia

Windows (PowerShell):

$env:HTTP_PROXY = "http://username:password@proxy.example.com:8080"
$env:HTTPS_PROXY = "http://username:password@proxy.example.com:8080"
insomnia

Note the URL format: even for HTTPS proxies, the http:// scheme is used, not https://. This is the standard convention for proxy environment variables. If the proxy does not require authentication, the URL will be shorter: http://proxy.example.com:8080.

Configuring SOCKS5 proxies

For SOCKS5 proxies, use the ALL_PROXY environment variable:

export ALL_PROXY="socks5://username:password@proxy.example.com:1080"

Insomnia supports SOCKS5, but not all versions handle SOCKS5 authentication correctly. If you encounter issues with authenticated SOCKS5, try using HTTP proxies or update Insomnia to the latest version.

Configuring via configuration file

For permanent proxy configuration, you can edit the Insomnia configuration file. The file location depends on the operating system:

• Windows: %APPDATA%\Insomnia\config.json
• macOS: ~/Library/Application Support/Insomnia/config.json
• Linux: ~/.config/Insomnia/config.json

Open the file in a text editor and add the proxy section:

{
  "proxy": {
    "http": "http://username:password@proxy.example.com:8080",
    "https": "http://username:password@proxy.example.com:8080"
  }
}

After editing, restart Insomnia. The proxy settings will automatically apply to all requests.

Using proxies in cURL: command examples

cURL is a classic command-line tool for working with HTTP, installed on almost all Unix-like systems and Windows 10+. It supports proxies via the -x or --proxy parameter. Let's explore various usage scenarios.

Basic usage of HTTP proxies

The simplest example of using an HTTP proxy without authentication:

curl -x http://proxy.example.com:8080 https://api.example.com/users

The -x parameter specifies the address and port of the proxy server. cURL will automatically determine that this is an HTTP proxy and configure the connection. If you want to explicitly specify the protocol, use the full URL:

curl --proxy http://proxy.example.com:8080 https://api.example.com/users

Proxies with authentication

If the proxy requires authentication with a username and password, add them to the proxy URL or use the -U parameter:

# Method 1: username and password in URL
curl -x http://username:password@proxy.example.com:8080 https://api.example.com/users

# Method 2: -U parameter (more secure, not saved in command history)
curl -x http://proxy.example.com:8080 -U username:password https://api.example.com/users

The second method is preferable, as the username and password will not be saved in the shell command history. For even greater security, you can specify only the username, and cURL will prompt for the password interactively:

curl -x http://proxy.example.com:8080 -U username https://api.example.com/users
# cURL will prompt for the password: Enter proxy password for user 'username':

Using SOCKS5 proxies

For SOCKS5 proxies, explicitly specify the protocol in the URL:

# SOCKS5 without authentication
curl -x socks5://proxy.example.com:1080 https://api.example.com/users

# SOCKS5 with authentication
curl -x socks5://username:password@proxy.example.com:1080 https://api.example.com/users

# SOCKS5h (DNS resolution through proxy)
curl -x socks5h://proxy.example.com:1080 https://api.example.com/users

The difference between socks5:// and socks5h://: in the first case, DNS resolution occurs locally (on your machine), while in the second case, it occurs through the proxy server. Use socks5h:// if you want to completely hide DNS queries from your provider.

POST requests through proxy

The proxy works the same for all HTTP methods. Here is an example of a POST request with JSON data through a proxy:

curl -x http://proxy.example.com:8080 \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"name":"John","email":"john@example.com"}' \
  https://api.example.com/users

To send files, use the -F parameter:

curl -x http://proxy.example.com:8080 \
  -X POST \
  -F "file=@document.pdf" \
  -F "description=Important document" \
  https://api.example.com/upload

Using environment variables

Instead of specifying the proxy in each command, you can use environment variables. cURL automatically reads http_proxy, https_proxy, and all_proxy:

# Set environment variables
export http_proxy="http://username:password@proxy.example.com:8080"
export https_proxy="http://username:password@proxy.example.com:8080"

# Now all cURL commands automatically use the proxy
curl https://api.example.com/users

# Disable proxy for a specific command
curl --noproxy "*" https://api.example.com/users

The --noproxy parameter allows you to exclude specific domains from proxying. For example, to not use the proxy for localhost and internal domains:

export no_proxy="localhost,127.0.0.1,.internal.company.com"

Configuring proxies in HTTPie

HTTPie is a modern alternative to cURL with human-readable syntax and colored output. It is especially convenient for interactive work with APIs in the terminal. HTTPie supports proxies through the --proxy parameter or environment variables.

Basic usage of proxies

The syntax for HTTPie for proxies is slightly different from cURL. You need to specify the protocol and proxy URL separately:

# HTTP proxy
http --proxy=http:http://proxy.example.com:8080 \
     --proxy=https:http://proxy.example.com:8080 \
     GET https://api.example.com/users

# Shortened form (if the proxy is the same for HTTP and HTTPS)
http --proxy=all:http://proxy.example.com:8080 GET https://api.example.com/users

Format: --proxy=protocol:proxy_url. For HTTPS requests, HTTP proxies (not HTTPS) are usually used; this is standard behavior.

Proxies with authentication

For authentication, add the username and password to the proxy URL:

http --proxy=all:http://username:password@proxy.example.com:8080 \
     GET https://api.example.com/users

HTTPie also supports the --proxy-auth parameter for more explicit credential specification, but in recent versions, this parameter is considered deprecated, and it is recommended to include authentication in the URL.

SOCKS5 proxies in HTTPie

HTTPie supports SOCKS5 proxies starting from version 2.0.0. Make sure you have the latest version installed:

# Check HTTPie version
http --version

# Update HTTPie via pip
pip install --upgrade httpie

# Use SOCKS5 proxy
http --proxy=all:socks5://proxy.example.com:1080 GET https://api.example.com/users

For authenticated SOCKS5, you will need to install the additional library pysocks:

pip install pysocks
http --proxy=all:socks5://username:password@proxy.example.com:1080 \
     GET https://api.example.com/users

POST requests and sending data

HTTPie has a convenient syntax for POST requests. Here is an example of sending JSON through a proxy:

# POST with JSON (HTTPie automatically determines Content-Type)
http --proxy=all:http://proxy.example.com:8080 \
     POST https://api.example.com/users \
     name="John Doe" \
     email="john@example.com" \
     age:=30

# Sending a file
http --proxy=all:http://proxy.example.com:8080 \
     POST https://api.example.com/upload \
     file@document.pdf \
     description="Important document"

Note the syntax: key=value for strings, key:=value for numbers and boolean values, key@file for files.

Using environment variables

Like cURL, HTTPie reads the standard environment variables http_proxy and https_proxy:

export http_proxy="http://username:password@proxy.example.com:8080"
export https_proxy="http://username:password@proxy.example.com:8080"

# Now all commands automatically use the proxy
http GET https://api.example.com/users

To temporarily disable the proxy, use the --proxy= (empty value) parameter:

http --proxy= GET https://api.example.com/users

Proxy authentication: username and password

Most commercial proxy services require authentication to prevent unauthorized use. There are two main methods of authentication: Basic Authentication (username/password) and IP Whitelist (IP address binding). Let's examine the features of each method and how to properly configure them in API clients.

Basic Authentication (username and password)

Basic Authentication is the most common method. The proxy server requires a username and password for each connection. The credentials are sent in the Proxy-Authorization header in Base64 format. All modern API clients automatically generate this header when you specify the username and password in the proxy URL.

The URL format with authentication:

protocol://username:password@proxy_host:proxy_port

Examples for different protocols:

# HTTP proxy
http://user123:pass456@proxy.example.com:8080

# HTTPS proxy (same format used)
http://user123:pass456@proxy.example.com:8080

# SOCKS5 proxy
socks5://user123:pass456@proxy.example.com:1080

IP Whitelist (IP binding)

Some proxy providers offer authentication by IP address. In this case, you add your IP to the whitelist in the proxy management panel, and the server allows connections only from this IP without a username and password. This is more convenient and secure, as you do not need to pass credentials with each request.

When using IP Whitelist, the proxy URL format is simplified:

# Without username and password
http://proxy.example.com:8080

# Example in cURL
curl -x http://proxy.example.com:8080 https://api.example.com/users

The downside of this method is that if your IP is dynamic (changes when reconnecting to the internet), you will have to update the whitelist each time. For servers with a static IP, this is the ideal option.

Special characters in passwords

If the proxy password contains special characters (@, :, /, ?), they need to be encoded in URL encoding (percent encoding). For example:

Example of using a password with special characters:

# Original password: P@ss:w0rd/123
# Encoded password: P%40ss%3Aw0rd%2F123

curl -x http://username:P%40ss%3Aw0rd%2F123@proxy.example.com:8080 \
     https://api.example.com/users

For automatic encoding, you can use online URL encoder tools or a command in Python:

python3 -c "import urllib.parse; print(urllib.parse.quote('P@ss:w0rd/123'))"
# Output: P%40ss%3Aw0rd%2F123

Troubleshooting common issues and errors

When working with proxies in API clients, common errors often arise. Let's discuss the most common problems and how to resolve them.

Error "407 Proxy Authentication Required"

This error means that the proxy server requires authentication, but the credentials were not provided or are incorrect. Solutions:

• Check the correctness of the username and password in the proxy provider's management panel
• Ensure that special characters in the password are encoded (see the section above)
• Check the URL format: http://username:password@host:port
• If using IP Whitelist, ensure that your current IP is added to the whitelist

Example of diagnosing in cURL with detailed output:

curl -v -x http://username:password@proxy.example.com:8080 https://api.example.com/users
# The -v (verbose) flag will show all headers and the authentication process

Error "Connection timeout" or "Failed to connect"

This error occurs when the API client cannot establish a connection to the proxy server. Possible reasons:

• Incorrect address or port of the proxy server — check the settings with the provider
• The proxy server is unavailable or overloaded — try another server from the list
• Your firewall is blocking outgoing connections to the proxy port
• The proxy provider has blocked your IP for exceeding limits

For diagnostics, try connecting to the proxy directly via telnet or nc:

# Checking proxy availability
telnet proxy.example.com 8080
# or
nc -zv proxy.example.com 8080

# If the connection cannot be established, the problem is in the network or the proxy server is unavailable

Error "SSL certificate problem"

When using HTTPS proxies, issues may arise with SSL certificate verification. This is especially relevant for MITM (Man-in-the-Middle) proxies that intercept and decrypt HTTPS traffic. Solutions:

• In cURL, use the -k or --insecure flag to disable certificate verification (for testing only!)
• In Postman, disable "SSL certificate verification" in Settings → General
• Install the proxy's root certificate in the system (for production environments)

# Disable SSL verification in cURL
curl -k -x http://proxy.example.com:8080 https://api.example.com/users

Best practices for working with proxies in API clients

To ensure smooth operation when using proxies in API clients, consider the following best practices:

• Always use HTTPS proxies when dealing with sensitive data to ensure encryption.
• Regularly check and update your proxy settings to avoid connection issues.
• Use environment variables for sensitive information like usernames and passwords to avoid hardcoding them in your scripts.
• Monitor your proxy usage to avoid exceeding limits set by your proxy provider.
• Test your API requests both with and without proxies to understand their impact on performance and behavior.

Conclusion

Configuring proxies in REST API clients can greatly enhance your testing and development capabilities. By understanding the types of proxies available and how to set them up correctly in tools like Postman, Insomnia, cURL, and HTTPie, you can effectively manage your API requests while ensuring security and performance. Following best practices will help you avoid common pitfalls and make the most out of your API interactions.