TCP Half-open Queue and Full Connection Queue

TCP Half-open Queue and Full Connection Queue

Description
The TCP half-open queue (SYN queue) and full connection queue (Accept queue) are two crucial queues maintained by the operating system's TCP protocol stack to implement the three-way handshake. Understanding these queues is essential for diagnosing server connection issues, analyzing network performance bottlenecks, and defending against SYN Flood attacks.

Detailed Explanation

  1. Origin of the Queues: Review of the Three-Way Handshake
    To understand these queues, we must first review the TCP three-way handshake process:

    • Step 1: The client sends a SYN segment to the server, indicating a request to establish a connection.
    • Step 2: Upon receiving the SYN, the server replies with a SYN-ACK segment, agreeing to establish the connection.
    • Step 3: After receiving the SYN-ACK, the client sends back an ACK segment, formally establishing the connection.

    The issue is that between the server sending the SYN-ACK and receiving the final ACK from the client, there is a time window. During this period, the connection is in an "intermediate state." To manage these connections in an intermediate state, the operating system creates the "half-open queue."

  2. Half-open Queue (SYN Queue)

    • Definition: After receiving the client's SYN segment and replying with SYN-ACK, the server places this connection request into a dedicated queue, which is the half-open queue. At this point, the connection state is SYN_RCVD.
    • Purpose: Temporarily stores connection information for connections that have completed the first and second handshakes but not yet the third handshake.
    • Queue Length Limit: The queue length is limited and determined by operating system parameters (in Linux, typically /proc/sys/net/ipv4/tcp_max_syn_backlog). Setting the queue length prevents resource exhaustion.
    • Common Issue - SYN Flood Attack: Malicious attackers spoof a large number of IP addresses, sending only the first handshake SYN packets but not replying with the third handshake ACK. This quickly fills the server's half-open queue, preventing legitimate connection requests from entering the queue, thereby achieving a denial-of-service attack. To counter this, Linux provides the syncookies mechanism. When syncookies is enabled, if the half-open queue is full, the server calculates a special cookie value based on the SYN packet and sends it in the SYN-ACK segment without allocating space in the queue. Only when the server receives an ACK segment carrying a valid cookie does it actually allocate resources to establish the connection, thus bypassing the queue length limitation.

  3. Full Connection Queue (Accept Queue)

    • Definition: After receiving the client's third handshake ACK segment, the kernel completes the connection establishment, removes this established connection from the half-open queue, and places it into another queue, which is the full connection queue. At this point, the connection state is ESTABLISHED.
    • Purpose: Stores connections that have completed the three-way handshake but have not yet been retrieved by the server application (e.g., Nginx, Apache) via the accept() system call.
    • Queue Length Limit: The length of the full connection queue is determined by the smaller of two parameters:
      • net.core.somaxconn: The system-wide maximum queue length parameter.
      • The backlog parameter passed by the application when calling the listen() function: For example, listen 80 backlog=2048; in Nginx configuration.
    • Common Issue - Queue Overflow: If the application processes connections slower than new connections are established, the full connection queue will fill up. When the queue is full, how the server kernel handles new connection requests depends on the system's TCP implementation (e.g., Linux's net.ipv4.tcp_abort_on_overflow parameter):
      • If set to 0 (default), the server simply ignores the ACK segment sent by the client, pretending not to have received it. After some time, the client will retransmit the ACK due to timeout because it receives no response, giving the server a chance to recover.
      • If set to 1, the server directly replies with an RST segment to reset the connection, which causes the client to report an error (e.g., "Connection reset by peer").

Summary and Analogy

You can think of these two queues as a bank's service process:

  • The half-open queue is like "taking a number and queuing." The customer takes a number (sends SYN), the manager gives you a waiting number (replies with SYN-ACK), and you wait in the waiting area. This waiting area is the half-open queue.
  • The full connection queue is like the "service waiting area." When your number is called (three-way handshake completed), you move from the waiting area to a seat in front of the counter (enter the full connection queue), waiting for the teller (the application) to officially serve you (call accept()).

By understanding these two queues, you can more deeply analyze connection timeout, connection reset, and other issues that may occur on servers under high concurrency scenarios, and know how to optimize by adjusting system parameters and application configurations.