Content Negotiation and Encoding

How negotiation works

The client states preferences, the server picks a representation and reports what it chose. Every negotiated dimension must appear in Vary, or caches will serve the wrong variant.

If the server cannot satisfy Accept, the correct response is 406 Not Acceptable. In practice most APIs ignore that and return their only format, which is tolerable but not spec-correct.

Quality values

q ranges from 0 to 1 with at most three decimals, defaults to 1, and q=0 means explicitly unacceptable.

Accept: text/html, application/xhtml+xml, application/xml;q=0.9, */*;q=0.8
Accept-Language: en-GB, en;q=0.9, fr;q=0.7, *;q=0.1
Accept-Encoding: br;q=1.0, gzip;q=0.8, *;q=0.1
Accept: application/json, text/plain;q=0

Specificity breaks ties before q does. text/html beats text/*, which beats */*, regardless of the quality values attached to the broader patterns.

Media types you will actually set

*Always send a charset for `text/`.** Without it browsers guess, and a guessed encoding is both a rendering bug and an XSS vector (UTF-7 style attacks).

Getting Content-Type wrong on JSON is the most common API bug of all, since a body sent as text/plain makes many frameworks leave req.body empty and also dodges the CORS preflight, which then surprises you later.

Compression

GET /app.js HTTP/1.1
Accept-Encoding: br, gzip

HTTP/1.1 200 OK
Content-Type: application/javascript; charset=utf-8
Content-Encoding: br
Vary: Accept-Encoding

curl --compressed -sI https://example.com/app.js | grep -i content-encoding
curl -sH 'accept-encoding: gzip' https://example.com/ -o - | gunzip | head
curl -s -o /dev/null -w '%{size_download}\n' https://example.com/app.js
curl -s -o /dev/null -w '%{size_download}\n' --compressed https://example.com/app.js

Rules that matter in production:

- Content-Encoding vs Transfer-Encoding: the first is a property of the resource and survives caching, the second is a property of one hop's framing. Compress with Content-Encoding. - Do not compress already-compressed formats. JPEG, PNG, WebP, MP4, and zip gain nothing and cost CPU. - Compressing secrets alongside attacker-controlled input leaks them through size, which is the BREACH and CRIME class of attack. Do not compress responses that reflect user input next to a CSRF token. - Pre-compress static assets at build time (.br and .gz next to the original) so you serve the highest ratio without per-request CPU. - Content-Length refers to the encoded body, so it shrinks with compression.

Charsets and text encoding

UTF-8 is the only sane answer. Explicit places to declare it:

Content-Type: text/html; charset=utf-8

<meta charset="utf-8">

Content-Disposition: attachment;
  filename="naive.pdf";
  filename*=UTF-8''na%C3%AFve.pdf

The plain filename is the ASCII fallback and filename* is what modern clients use, so send both.

API versioning through negotiation

curl -H 'Accept: application/vnd.example.v2+json' https://api.example.com/orders

Whichever you pick, the response must echo its own type in Content-Type and add the negotiated header to Vary. Path versioning wins on operational simplicity, media-type versioning wins on purity, and mixing both in one API is the option to avoid.

Language negotiation and the Vary trap

GET /docs HTTP/1.1
Accept-Language: es-MX, es;q=0.9, en;q=0.5

HTTP/1.1 200 OK
Content-Language: es
Vary: Accept-Language
Content-Type: text/html; charset=utf-8

Language matching falls back by prefix, so es-MX should fall back to es before it falls back to en. Serve a default when nothing matches rather than returning 406.

The trap: Accept-Language has enormous variety across real clients, so varying on it fragments your cache badly. Most sites are better off with distinct URLs per language (/es/docs, /en/docs) plus a one-time redirect based on the header, which is also better for search indexing. Reserve true negotiation for APIs, where the client population is narrow.

curl -sI -H 'accept-language: es' https://example.com/docs | \
  grep -iE 'content-language|vary|location'
curl -sI -H 'accept: application/xml' https://api.example.com/v1/orders | head -1