Migration of Network Protocols to Post-Quantum Cryptography (PQC): Performance Challenges, Key Size Issues, and Hybrid Solutions in TLS 1.3, IPsec, and QUIC

Migration of Network Protocols to Post-Quantum Cryptography (PQC): Performance Challenges, Key Size Issues, and Hybrid Solutions in TLS 1.3, IPsec, and QUIC

Mohammad Iman Golzari1 Rezvan Sharifi2 Amir Hosein Asady3 Muhammad Hosein Kalali4 Behzad Molvai5 Mostafa Farhadi-Moghadam6

1) Computer Science Student, Department of Computer, Vahdat-Torbat Jam Institute of Higher Education, Email:
2) Computer Science Student, Department of Computer, Vahdat-Torbat Jam Institute of Higher Education, Email:
3) Computer Science Student, Department of Computer, Vahdat-Torbat Jam Institute of Higher Education, Email:
4) Department of Computer, Vahdat-Torbat Jam Institute of Higher Education, Email:
5) Corresponding Author, Department of Computer Science, Vahdat-Torbat Jam Institute of Higher Education, Email:
6) Department of Computer Science, Vahdat-Torbat Jam Institute of Higher Education, Email:

IOI Code : AAHD-ABHHKC
Publication : National Conference on Information Technology, Nanotechnology, Artificial Intelligence and Technological Futures Studies(itnaf.ir)
Abstract :
    The rapid progress toward cryptographically relevant quantum computers poses an existential threat to the public-key primitives, such as RSA and ECC, that secure TLS 1.3, IPsec, and QUIC, enabling harvest-now-decrypt-later attacks on long-lived network traffic. This paper presents a comprehensive cross-protocol performance evaluation of post-quantum cryptography migration using the NIST-standardized algorithms ML-KEM and ML-DSA under realistic network conditions. We compare classical, hybrid, and pure post-quantum configurations across TLS 1.3, IPsec/IKEv2, and QUIC. Experiments were conducted on production-grade implementations, including OpenSSL with liboqs and strongSwan, under controlled latency ranging from 0 to 50 ms, bandwidth from 100 Mbps to 10 Gbps, packet loss from 0 to 5 percent, and MTU constraints. More than 1,000 iterations were performed for each configuration, with statistical reporting of median, p95, and p99 latencies. The results show that hybrid key exchange introduces only modest overhead in TLS 1.3, with an 8–12 percent increase in median handshake latency and approximately 1.4–1.6 KB of additional data transmitted over the network. TLS 1.3 is also largely unaffected by fragmentation because it operates over TCP. In contrast, UDP-based protocols experience more significant performance issues. IPsec shows a 65–72 percent fragmentation probability without mitigation, which is reduced to below 8 percent through IKE_INTERMEDIATE, at the cost of a 22–35 percent increase in latency. QUIC exhibits a 15–24 percent increase in latency, which rises to 31–42 percent under packet loss. Pure post-quantum configurations consistently underperform hybrid approaches. Energy measurements on edge-class hardware and high-concurrency server tests further highlight the practical constraints of post-quantum migration.

Keywords : post quantum cryptography tls 1 3 ipsec quic hybrid key exchange ml kem fragmentation network protocol migration