HTTP/2: Binary Framing, HPACK, Stream Multiplexing

Why HTTP/2

HTTP/1.1 (RFC 7230) is text-based, has head-of-line blocking at the request level, and does not allow parallel requests over one connection. Browsers opened 6 parallel TCP connections per host: 6 TLS handshakes, duplicated headers, and poor use of the congestion window.

In 2015 the IETF published HTTP/2 (RFC 7540, reissued as RFC 9113):

• Binary - not human-readable, but faster and unambiguous to parse
• One TCP connection per host - fewer handshakes, better congestion window
• Stream multiplexing - parallel requests as independent streams
• HPACK - header compression through a shared dictionary
• Server push (now deprecated)

The relationship to HTTP/1.1 is not replacement but transport optimization. Semantics (method, URI, status codes, body) are unchanged. Only the wire format differs.

In 2020, [[quic-http3|HTTP/3]] arrived: same multiplexing, but over QUIC/UDP, without TCP HoL.

Binary framing layer

Every HTTP/2 communication is split into frames (the minimum unit). The frame header is 9 bytes:

 0                   1                   2                   3

 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

|                 Length (24)                   |

+---------------+---------------+---------------+

|   Type (8)    |   Flags (8)   |

+-+-------------+---------------+-------------------------------+

|R|                 Stream Identifier (31)                      |

+=+=============================================================+

|                   Frame Payload (0...)                      ...

+---------------------------------------------------------------+

• Length - payload size (max 2^14 = 16384 bytes by default, can be increased to 2^24)
• Type - frame type (see below)
• Flags - bit-flags dependent on type
• Stream Identifier - 0 for control frames, otherwise the stream ID

Frame types

Streams: independent virtual connections

Each request/response pair is a stream with a unique 31-bit ID. Streams are independent: frames from different streams can interleave within the same TCP connection.

Stream lifecycle:

idle → reserved (server push) ─┐

     └→ open ──┐                │

               ├→ half-closed ──┼→ closed

               └→ half-closed ──┘

ID rules:

• Even IDs are server-initiated (server push)
• Odd IDs are client-initiated
• IDs only increase; reuse is forbidden

One TCP connection can carry thousands of streams (limited by SETTINGS_MAX_CONCURRENT_STREAMS).

HPACK: header compression

HTTP headers repeat: User-Agent, Accept-Encoding, Cookie are identical on every request. HTTP/1.1 sends them as plaintext each time.

HPACK (RFC 7541) compresses them through:

1. Static table - 61 predefined headers (:method GET, content-type text/html, ...)
2. Dynamic table - extensible, populated from previous requests
3. Huffman coding - compression of literal values

You transmit a table index (1-2 bytes) instead of the full text (50+ bytes). Typical header compression is 80-90%.

Downside: the dynamic table requires per-connection state, synchronized between both sides. CRIME/HEIST-style attacks call for caution with user-controlled headers (cookie compression).

HTTP/3 uses QPACK (an adaptation of HPACK for QUIC): same idea, but without head-of-line blocking on decompression.

Stream priority, and why it is deprecated

HTTP/2 (RFC 7540) introduced a complex priority tree: each stream has a parent stream, a weight (1-256), and an exclusive flag. The intent was for servers to process critical streams (HTML before CSS before images) first.

In practice:

• Browsers implemented it differently from one another
• Servers often ignored it
• Race conditions appeared in the dependency tree

RFC 9113 (2022) deprecated priority. The new RFC 9218 introduced Extensible Priorities: simple Priority: headers with the scheme urgency=N, incremental=?1.

Flow control: per-stream and per-connection

HTTP/2 has credit-based flow control (similar to the TCP receive window):

• Each sender knows the receiver's current window size
• It sends DATA frames while window > 0
• The receiver sends WINDOW_UPDATE to grant more credit

Two levels: per-stream and per-connection. Without credit you cannot send DATA; HEADERS frames are always allowed (they are control-plane traffic).

The default window of 65535 bytes is very small: at 100 ms RTT it gives roughly 5 Mbps per connection. Servers typically send an immediate WINDOW_UPDATE to raise the window to 16-64 MB.

Server push: deprecated

The idea was that the server would send resources before the client asked (HTML together with the linked CSS). RFC 7540 defined it, and many servers implemented it.

In practice:

• Browsers already had the resource cached, wasting bandwidth
• Deciding what to push was hard to get right
• 103 Early Hints covers the use case better

Chrome disabled server push support in 2022. RFC 9113 (2022) marked PUSH_PROMISE as deprecated.

The replacement is 103 Early Hints:

HTTP/2 103 Early Hints

Link: </styles.css>; rel=preload; as=style

Link: </app.js>; rel=preload; as=script

HTTP/2 200 OK

Content-Type: text/html

...

The browser starts fetching the preloads while the server generates the main response.

TCP head-of-line blocking: the problem

The big caveat of HTTP/2 is one TCP connection, one in-order byte stream. If a single packet is lost, all streams wait for the retransmit:

Stream 1: [seg1][seg2][LOST][seg4]   <- Stream 1 waits for seg3

Stream 2: [data][data][data][data]   <- Stream 2 waits too

Stream 3: [data][data][data][data]   <- And Stream 3

Even though Stream 2 data is already in the NIC buffer, the kernel will not deliver it to the application until seg3 arrives (TCP delivers in order). This is TCP HoL blocking.

On a lossy Wi-Fi or 4G link this erases the multiplexing advantage. HTTP/1.1 with 6 connections is sometimes faster in those conditions.

The fix is [[quic-http3|HTTP/3 / QUIC]]: streams in QUIC are independent at the UDP level, so loss on one stream does not block the others.

Negotiation: how the client learns that the server supports h2

Two mechanisms:

ALPN in TLS (standard)

In the TLS ClientHello the client sends ALPN (Application-Layer Protocol Negotiation) with a list: h2, http/1.1. The server selects one in the ServerHello. Browsers use only this path. HTTP/2 in clear text is not supported by any browser.

Upgrade header (h2c, plaintext)

GET / HTTP/1.1

Host: example.com

Connection: Upgrade, HTTP2-Settings

Upgrade: h2c

HTTP2-Settings: <base64-encoded SETTINGS>

If the server supports h2c it replies with 101 Switching Protocols. This path is used only for server-to-server communication (gRPC commonly). RFC 9113 removed the Upgrade mechanism, though many servers and clients still support it.

HTTP/2 vs HTTP/3

When things go wrong

• HTTP/2: protocol error - a frame with a malformed format or a state violation. Use tcpdump and the Wireshark HTTP/2 dissector to identify the exact frame. Often a bug in a client or server stub.
• GOAWAY with last_stream_id - the server is closing the connection. Requests with stream_id > last must be retried on a new connection. Frequent cause: rate limiting or idle timeout.
• Window starvation - the application is not sending WINDOW_UPDATE, so the sender stalls. Check the library and increase buffer sizes.
• Plaintext h2c does not work in browsers - all browsers require TLS + ALPN. Use --http2-prior-knowledge in curl for local testing.
• HPACK decompression bomb - an implementation may be vulnerable to DoS if the client sends many new headers, bloating the dynamic table. Limits exist in the RFC, but implementations vary (CVE-2019-9518).
• Throughput worse than HTTP/1.1 - likely high packet loss with multiplexing blocked by TCP HoL. Switch to HTTP/3.
• SETTINGS_MAX_CONCURRENT_STREAMS = 100 - a common server default. If the client sends more streams they queue up and latency climbs. Raise the limit or use a connection pool.