RUSTSEC-2026-0339: Unbounded growth of pending connection ID retirements in quinn-proto #461

Unbounded growth of pending connection ID retirements in quinn-proto

When processing a NEW_CONNECTION_ID frame, quinn-proto queues a RETIRE_CONNECTION_ID frame for
each connection ID that the peer asked it to retire. The normal path bounds this queue and closes
the connection with a CONNECTION_ID_LIMIT_ERROR if the bound would be exceeded. However, the path
handling a connection ID that had already been retired pushed onto the queue without applying
the limit and without deduplicating.

A peer that has completed the handshake can first send a NEW_CONNECTION_ID frame with a large
retire_prior_to value to advance the receiver's connection ID window, and then repeatedly send
NEW_CONNECTION_ID frames with sequence numbers below that window. Each such frame adds another
entry to the queue. The queue is only drained by sending RETIRE_CONNECTION_ID frames, which are
subject to congestion control, so a peer that withholds acknowledgements can stall the drain while
continuing to flood, growing the victim's memory use without bound.

The flaw was corrected in 0.11.17 by applying the same limit on the already-retired path.

Thanks to @scadastrangelove for reporting this issue.

See advisory page for additional details.