Skip to content

Security

An API client holds credentials and follows URLs it did not write: redirects, cursor URLs, links in a response body. headgate makes the safe behavior the default and loosening it explicit.

At a glance

Threat What headgate does Your control
a credential sent to another host by a redirect or a link withholds it on every hop to an origin you did not trust trust more origins with trusted_origins=
an https page redirecting to plain http refuses the redirect always on
a URL that reaches your internal network (SSRF) refuses private addresses, with DNS pinning, once you turn the guard on allow_private_hosts=False turns it on
a value like ../admin rewriting the path escapes each value as one segment {value:raw} for a pre-built fragment
a token printed in a log or error redacts it as *** in every error and event always on
a proxy or .netrc picked up from the environment ignores the environment pass proxy= or ssl_context= explicitly
a huge response exhausting memory caps a buffered body at 32 MiB max_response_bytes=
one user's cached answer served to another partitions the cache by credential Partition.SHARED for public data

Every hop goes through the same checks, not only the first request: each redirect, each next page and each request an auth flow sends.

  1. Origin checknot your origin or a trusted one? Credentials, Authorization and Cookie are removed, and it goes out unsigned
  2. Address checkwith the SSRF guard on, every address the host resolves to must be public, or nothing is sent
  3. Pinned connectthe connection goes to the address that was checked, so DNS cannot swap it
  4. Response checkan https-to-http redirect is refused, and a one-shot body is never sent twice

Credentials stay on your origin

Every auth headgate ships declares the headers and query parameters it stamps (the CredentialStamper protocol: credential_headers, credential_params, credential_identity()). A request hook runs before every hop: the first request, each redirect, and each request an auth flow sends. On any hop addressed to an origin that is neither base_url's nor in HttpOptions(security=SecurityOptions(trusted_origins=...)), it removes:

  • every header and query parameter the auth declares;
  • Authorization and Proxy-Authorization, whatever set them, so a third-party auth or a caller's own header does not travel either;
  • Cookie, unless the client keeps a cookie jar (cookies=True), which scopes cookies to their domains itself.

The request still goes out, unsigned, so a signed download URL on a CDN works. A credential meant for another origin needs that origin in trusted_origins, written as a bare origin:

from headgate import HttpOptions, SecurityOptions

http = HttpOptions(
    security=SecurityOptions(
        trusted_origins=frozenset({"https://uploads.example.com", "https://eu.api.example.com:8443"})
    )
)

The hops a CachedTokenAuth mint flow yields may carry the mint's credentials to the token endpoint its code addressed, and to the redirects that endpoint answers with on the same origin, and nowhere else.

A middleware that stamps a credential (a request signer) declares it the same way, with credential_headers and credential_params. A middleware runs inside the transport stack, after the request hook, so the guard runs a second time below the last middleware, on every attempt of every hop, and removes what it declares from any hop not addressed to base_url's origin or a trusted one. An auth flow's token host is not trusted for a middleware's credentials: the middleware did not choose it.

A third-party httpx2.Auth that is not a CredentialStamper, or a middleware that declares nothing, and puts its credential in another header or in the query is not covered; declare the names to have them withheld. A cache refuses such an auth unless told the data is public (see caching).

Redirects

  • No downgrade. A redirect from https to http is never followed: the 3xx becomes the call's final response and raises as the UpstreamStatusError it is, with a note naming the refused target.
  • A hop limit. HttpOptions(max_redirects=5) is the default. An auth flow's hops count toward it too, because httpx2 keeps one history per call; a client whose mint takes several hops raises it. Past it the call raises TransportError ("Exceeded maximum allowed redirects").
  • No body sent twice. A 307 or 308 that would re-send a one-shot streamed body is not followed; it raises as its 3xx, with a note suggesting a body factory. A 303, and a 301 or 302 answering a POST, become a body-less GET as usual.
  • No unbounded redirect bodies. The body of a redirect being followed is dropped unread, so a large 302 cannot bypass the response size cap.
  • Off when you need it. HttpOptions(follow_redirects=False), or follow_redirects=False on a call, makes every redirect the final response: declare it with Expect(...).on(302, ...) or it raises.

The SSRF guard

A client that fetches URLs it did not write (webhooks, user-supplied links, cursors from a partner) sets HttpOptions(security=SecurityOptions(allow_private_hosts=False)). Every hop's host is resolved and refused, as ForbiddenDestinationError, when any address it resolves to is not globally routable: loopback, private, link-local, carrier-grade NAT, multicast, reserved or unspecified, with IPv4-mapped and NAT64 addresses judged by the IPv4 address they reach. Nothing is sent to a refused host, and the refusal is never retried or charged to a breaker.

The default transport pins each connection to the address it vetted, in the same step as connecting, so a DNS answer that changes between the check and the connect (DNS rebinding) cannot redirect it. The request keeps its host name, so TLS still sends it as SNI and verifies the certificate against it, and pooled connections are keyed by name.

  • A custom transport (HttpOptions(transport=...), the curl transport included) resolves names itself: every hop is still vetted before it is sent, but the transport's own lookup is not pinned.
  • The guard cannot be combined with proxy=, since a proxy resolves the target itself.
  • HttpOptions(security=SecurityOptions(resolver=...)) replaces the event loop's getaddrinfo (an async function of host and port returning addresses as text), and is accepted only with the guard on.

Paths built from values

A t-string path escapes every interpolated value as exactly one segment:

await client.delete(t"positions/{symbol}", expect=nothing)  # "BTC/USD" -> positions/BTC%2FUSD

So a value cannot change the URL's structure: /, ?, # and whitespace are percent-encoded, a value of . or .. is sent as %2E or %2E%2E rather than resolved as a dot segment, and an empty value raises ValueError rather than addressing the collection. A nested t-string splices as structure (t"{API_PREFIX}/accounts", its own values still escaped), and {fragment:raw} is the one explicit escape hatch for a pre-built fragment. A plain str URL is sent as it is, which is what lets a paginator follow an absolute URL from a response.

Redaction

Nothing headgate raises or reports carries a credential in clear:

  • every URL in an error message or event has its userinfo and secret query values replaced with ***. Secret names are the well-known ones (api_key, token, access_token, signature, X-Amz-Signature, session, and more, compared ignoring case, _ and -) plus every name a QueryParamAuth was given;
  • a header value the HTTP library refuses to encode is reported by the header's name, never its value;
  • events carry no headers and no bodies; error messages carry a bounded snippet of an error response's body, which is the upstream's own text.

QueryParamAuth puts the key in the URL, and URLs are not secret to every tool: a tracing integration that records full request URLs records it. Exclude such hosts from URL capture, or prefer a header.

Hermetic by default

Nothing is read from the environment: no proxy variables, no .netrc, no SSL_CERT_FILE, because the HTTP client and the default transport are built with trust_env=False. Cookies a response sets are neither stored nor sent back unless HttpOptions(cookies=True). TLS verification is always on; pass HttpOptions(ssl_context=...) for a private CA, mutual TLS or pinning, and proxy= (http, https or socks5 URL) for a proxy. Those three configure the default transport, so combining them with transport= is a ValueError.