Transport Thresholds and Network-Impairment Amplification Mechanisms in TLS 1.3 Post-Quantum Key Exchange

Post-quantum key-encapsulation mechanisms substantially enlarge TLS 1.3 handshake material, but their end-to-end impact depends on the combined effects of TLS encapsulation, TCP segmentation, and network conditions. Using OpenSSL 3.0.21, liboqs 0.16.0, and oqs-provider 0.12.0-dev, this study conducts cross-layer experiments on X25519, ML-KEM-768, two implementation-specific HQC groups, hqc1 and hqc3 (not NIST security levels), and the corresponding hybrid schemes. It measures ClientHello, ServerHello, TLS record, and TCP carriage structures; locates segmentation-transition boundaries through adjacent-MTU scans; and completes 180,000 formal connection-establishment measurements plus 6,400 targeted review measurements in separate RTT, per-direction bandwidth, and independent per-direction loss experiments. The results show that ML-KEM is generally close to X25519 when there is no packet loss or material bandwidth bottleneck. At 1 Mbps, the latency increase of each scheme is broadly consistent with serialization estimates derived from bidirectional TCP payload. Packet loss first amplifies P95 and P99; under 10% independent loss in each direction, 30 of 18,000 measurements did not complete and verify the connection within 10 s. The higher median latency and long tail of HQC-L3 recur in both the formal batches and targeted review, whereas the single-batch anomaly of X25519-HQC-L1 does not. The study establishes a traceable evidence chain from cryptographic material, TLS encapsulation, and TCP segmentation to network impairment and connection-establishment performance, providing experimental support for MTU/MSS headroom assessment and post-quantum TLS deployment.