HTTP Headers Reference

Comprehensive reference for http headers reference covering essential concepts, practical examples, and production best practices.

Orientation

HTTP headers are the metadata that travels alongside every request and response in the web stack. They govern everything from authentication and caching to content type negotiation and protocol version. This matters whenever you’re debugging a connectivity issue, tuning performance, or implementing security controls—and it’s especially critical when things break in production with no obvious error message. This reference is organized by what you’re trying to accomplish, not by header name taxonomy. It includes exact commands, configuration patterns, and the failure modes you only learn after a midnight pager duty call.

Modifying Request Headers

Adding or Overwriting a Single Header

The most common use case is injecting a header into an outgoing request. curl provides the canonical mechanism:

curl -H "X-Request-Id: 4f2a3c9d-8e7b-4f1a-b2d6-9e5c2a7f3d10" https://api.example.com/v1/resource

The -H (or --header) flag accepts Header-Name: Value. If the header already exists in the request, curl overwrites it. To append rather than replace (supported by some servers via -H "X-Foo: bar" -H "X-Foo: baz"), you send the flag twice—but be aware that HTTP semantics treat duplicate headers per the spec’s list-or-single-value rules; many servers collapse them to the first or last occurrence depending on implementation.

Supplying Headers from a File

When the header value is long, contains special characters, or you’re repeating the same set across many invocations, store them in a file and pass it with --header-file:

cat > extra_headers.txt <<'EOF'
X-Api-Version: 2024-03-15
User-Agent: my-tool/1.2.3
EOF

curl --header-file extra_headers.txt https://api.example.com/v1/resource

Each line in the file must follow Name: Value. Blank lines and lines starting with # are ignored. This flag is supported by curl 7.78.0 and later; earlier versions require --libcurl workflows or manual construction.

Removing a Header Sent by the Client

Sometimes you need to strip a header that the client automatically adds—for instance, suppressing User-Agent or removing a proxy-injected X-Forwarded-For. There’s no universal flag to delete a header; the pattern depends on the tool:

Changing the Host Header

The Host header is special—it’s the only header the HTTP/1.1 spec mandates the client send, and it’s how the server routes the request. In reverse-proxy setups, you may need to override it:

curl -H "Host: original-domain.com" https://edge-proxy.example.com/path

Note: this does not change the TCP-level Host portion of the TLS SNI or the destination IP/port. It only modifies the application-layer header. If you’re trying to reach a virtual host behind a load balancer, the proxy may reject or rewrite the header, and the back-end service may see a mismatch between the Host header and the TLS certificate—producing an SSL alert or 400 Bad Request.

Inspecting Response Headers

Verbose curl Output

To see both request and response headers, pass -v (verbose):

curl -v https://api.example.com/v1/resource

The output intersperses >, < markers with the transferred data. Headers sent by the client appear after >, and headers returned by the server appear after <. Look for the == Info \n == == Transfer \n == separator to delineate header sections from body.

To capture headers only (without the body), use -I (or --head):

curl -I https://api.example.com/v1/resource

This sends a HEAD request and prints the response headers. Note that some servers return a subset of headers in HEAD responses compared to GET; Content-Length may be absent, and ETag/Last-Modified may differ. If you need the exact headers a GET would produce, use -I only for metadata you know is safe to inspect head.

Filtering Specific Headers

When you only care about a subset—for example, checking X-Rate-Limit-Remaining or Cache-Control—pipe through grep:

curl -sI https://api.example.com/v1/resource | grep -i "x-rate-limit"

or, more precisely, use curl's --output and awk:

curl -sI https://api.example.com/v1/resource | awk -F': ' '/^X-Rate-Limit-Remaining: /{print $2}'

This prints only the value of that header, trimming the prefix. Be wary of headers with colon-containing values (e.g., Set-Cookie: session=abc:def); awk will capture everything after the first colon, so you may need a more specific pattern or dedicated parsing.

Authentication Patterns

Bearer Tokens

The de facto standard for API authentication is the Authorization header with a Bearer token:

curl -H "Authorization: Bearer $TOKEN" https://api.example.com/v1/resource

Replace $TOKEN with the actual JWT, opaque token, or personal access token. The word Bearer is case-insensitive per RFC 7235, but most implementations expect exactly Bearer. If the token contains characters that the shell interprets (spaces, $, backticks), quote it:

curl -H "Authorization: Bearer $(cat token.txt)" https://api.example.com/v1/resource

API Keys via Custom Headers

Many services accept an API key through a non-standard header, often X-Api-Key, X-Auth-Token, or Authorization: Key <value>. There’s no universal registry; you’ll need to consult the specific service’s documentation. A common pattern looks like:

curl -H "X-Api-Key: abc123xyz789" https://api.example.com/v1/resource

If the key is long or contains special characters, store it in an environment variable and reference it:

export API_KEY=abc123xyz789
curl -H "X-Api-Key: $API_KEY" https://api.example.com/v1/resource

HMAC Signatures

Some older or high-security APIs require an HMAC signature in headers, typically X-Signature, X-Signature-Timestamp, and X-Signature-Version. The client computes a hash of the request body plus a timestamp using a shared secret, then sends the result. There’s no generic command-line tool that does this automatically for arbitrary APIs; you’ll typically write a small script. For example, in bash with openssl:

TIMESTAMP=$(date +%s)
PAYLOAD=$(cat body.json)
SIGNATURE=$(echo -n "${TIMESTAMP}${PAYLOAD}" | openssl dgst -sha256 -hmac "$SHARED_SECRET" | awk '{print $NF}')

curl -H "X-Signature: ${SIGNATURE}" \
     -H "X-Signature-Timestamp: ${TIMESTAMP}" \
     -H "X-Signature-Version: v1" \
     -d "@body.json" https://api.example.com/v1/resource

The server recomputes the signature and compares it, rejecting the request if the timestamp is too old (a replay-prevention window, typically 5–15 minutes). If the signature mismatches, the server responds 401 Unauthorized or 403 Forbidden with no body in many implementations—making debugging harder without verbose logging.

Caching Control

Supplying Cache-Control

Proxies and browsers obey the Cache-Control header to determine how long to keep a response. When making a request, you can influence caching behavior:

curl -H "Cache-Control: no-cache" https://api.example.com/v1/resource

no-cache does not mean “don’t cache”; it means “reuse a cached response only after successful validation with the server (e.g., If-Modified-Since or ETag).” For absolute freshness:

curl -H "Cache-Control: no-store" https://api.example.com/v1/resource

no-store directs caches not to store any part of the response. Note that no-cache and no-store are directives on the response direction by default; supplying them on the request tells the client “I don’t want stale data,” but intermediate proxies may interpret them differently depending on their configuration.

If-Modified-Since and If-None-Match

When you have a known ETag or Last-Modified timestamp, you can use conditional requests to avoid transferring the full body:

curl -H "If-Modified-Since: Tue, 15 Mar 2024 12:00:00 GMT" \
     -H "If-None-Match: \"abc123def456\"" \
     https://api.example.com/v1/resource

If the resource hasn’t changed, the server responds 304 Not Modified with no body. If it has changed, the server responds 200 OK with the full resource and updated headers. A common failure mode: the If-Modified-Since header uses HTTP-date format (RFC 1123), and many clients send it with a timezone offset that the server rejects, resulting in 400 Bad Request. Always use the exact format: Wdy, DD Mon YYYY HH:MM:SS GMT.

Content Negotiation

Supplying Accept and Content-Type

The Accept header tells the server which response formats you can process:

curl -H "Accept: application/vnd.api+json; version=2" https://api.example.com/v1/resource

If the server can’t produce a matching representation, it may respond 406 Not Acceptable. Some APIs version via URL path or query parameter rather than Accept; the header is most common in RESTful APIs that follow content-type negotiation standards.

The Content-Type header describes the media type of the request body:

curl -H "Content-Type: application/json" -d '{"key":"value"}' https://api.example.com/v1/resource

If you omit Content-Type, curl defaults to application/x-www-form-urlencoded when -d is used, or sends no Content-Type when using -F (multipart form). Many APIs reject requests without an explicit Content-Type, returning 415 Unsupported Media Type.

Accept-Encoding and Compression

Servers may gzip or brotli-compress responses based on Accept-Encoding. To request compressed output:

curl -H "Accept-Encoding: gzip, deflate, br" https://api.example.com/v1/resource

The server will include Content-Encoding: gzip in the response and compress the body. Your curl version must support the encoding; most modern versions do. If the server doesn’t support any of the listed encodings, it returns the uncompressed response and omits Content-Encoding. A subtle edge: some proxies strip Accept-Encoding, causing the server to send uncompressed data, which then passes through the proxy uncompressed to the client—potentially exceeding bandwidth expectations.

Common Confusions and Silent Failures

Host vs. X-Forwarded-Host

Behind a reverse proxy (nginx, HAProxy, CloudFront), the original Host header arriving at the backend is often X-Forwarded-Host or X-Forwarded-For–derived. A frequent gotcha: an application reads Host and finds it set to the proxy’s hostname (e.g., cdn.example.com) instead of the original client domain. The fix is to configure the proxy to forward the original Host via X-Forwarded-Host, but not all proxies do this by default. If you’re seeing unexpected redirects or absolute URLs in responses, inspect what the backend actually receives with curl -v.

Content-Length Mismatch

When you -d a payload in curl, the client automatically sets Content-Length to the byte size of the data. If you also -H "Content-Length: 0" on the same request, curl may or may not overwrite the automatic value depending on version; in recent versions, the explicit -H takes precedence, but the behavior changed between curl 7.69 and 7.81. The safe pattern: don’t set Content-Length manually; let the library compute it.

Duplicate Headers

HTTP permits multiple headers with the same name in certain contexts (e.g., Set-Cookie), but many servers and proxies collapse them. If you -H "X-Foo: 1" -H "X-Foo: 2", some back ends see only X-Foo: 1 (first), others see X-Foo: 2 (last), and some return an error. If you need multiple values, use a comma-separated list if the API supports it (X-Foo: 1, 2) or switch to a header that natively supports lists (like Set-Cookie with multiple Set-Cookie responses, or Link headers).

Production Breakage Patterns

Header Truncation Proxies

In environments with strict proxy configurations (e.g., enterprise web proxies, API gateways), headers exceeding a certain byte limit (often 8KB–16KB) are silently dropped. If you’re sending a JWT in a custom header or a large X-Request-Id, the proxy may strip it without returning an error; the backend never sees it and authenticates based on a missing or default value. Test with curl -v and look for len= lines in the verbose output indicating header size. If you suspect truncation, fragment the header value or move the data to the request body/query string.

Case Sensitivity

HTTP headers are case-insensitive by RFC spec, but real-world implementations vary. curl preserves the case you type; the server may compare against lowercase keys. If you -H "X-Api-Key: abc" and the server expects x-api-key, it may fail to match. To avoid ambiguity, use all-lowercase or match the exact casing documented by the API. Some frameworks (Express.js, Django, Flask) normalize headers to CONTENT_TYPE, AUTHORIZATION, etc., uppercasing and replacing hyphens with underscores—if you’re writing code that reads headers, be aware of this normalization.

Missing Headers in HEAD Responses

As noted earlier, curl -I sends a HEAD request. Some servers omit headers that describe the body size or entity tags in HEAD responses, on the grounds that the client shouldn’t need them for a head operation. If you’re programmatically checking for Content-Length or ETag via a HEAD probe, you may get a 200 OK with no those headers, leading your logic to assume the resource is unchanged or non-existent. Use GET with Range: 0-0 or conditional If-Modified-Since/If-None-Match on a full request instead, if you need those headers reliably.

TLS-SNI Mismatch with Custom Host

When you -H "Host: internal-service.example.com" to curl toward a TLS-terminating load balancer, the TCP connection’s SNI (Server Name Indication) still uses the destination hostname you typed on the command line. If the certificate was issued for api.example.com but you override Host to internal-service.example.com, the TLS handshake may fail with schannel: SSL/TLS connection failure (Windows) or SSL alert: bad certificate (Linux). The Host header is application-layer; SNI is transport-layer. If you need both a custom Host and the correct SNI, you’ll need to use a tool that can set both, or terminate TLS at your edge and connect internally without encryption.

Summary Checklist

This page was rewritten on 14 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.