TCP Half-Open Connection Queue and Established Connection Queue

TCP Half-Open Connection Queue and Established Connection Queue

Description
During the TCP three-way handshake process, the server uses two important queues to manage connections in different states: the Half-Open Connection Queue (SYN Queue) and the Established Connection Queue (ACCEPT Queue). Understanding how these two queues work is crucial for analyzing server performance and diagnosing connection timeout or rejection issues.

Knowledge Point Explanation

1. Basic Concepts and Queue Roles

When a client sends a SYN packet to initiate a connection, the server, upon receiving it, places the connection into a dedicated queue. At this point, the connection is in the SYN_RCVD state. This queue is the Half-Open Connection Queue (or SYN Queue).

Simultaneously, the server replies with a SYN-ACK packet. If the client normally replies with an ACK packet to complete the three-way handshake, the server will remove the connection from the half-open queue and place it into another queue. The connection then enters the ESTABLISHED state, waiting for the application on the server (such as a web server) to call the accept() system call to retrieve and process it. This queue holding established connections is the Established Connection Queue (or ACCEPT Queue).

In simple terms:

  • Half-Open Connection Queue: Stores connections that have received a SYN but have not completed the three-way handshake (SYN_RCVD state).
  • Established Connection Queue: Stores connections that have completed the three-way handshake but have not been accept()ed by the application (ESTABLISHED state).

2. Detailed Process of Three-Way Handshake and Queue Interaction

Let's track step-by-step how a connection request interacts with these two queues:

  • Step 1: Client sends SYN

    • The client sends a SYN segment with a sequence number J, indicating a desire to establish a connection.
    • The server receives the SYN packet.

  • Step 2: Server responds with SYN-ACK and creates a half-open entry

    • The server kernel allocates resources for the connection, initializing a Transmission Control Block (TCB).
    • The server places the connection information (e.g., source/destination IP and port) into the Half-Open Connection Queue. The connection state is now SYN_RCVD.
    • The server sends a SYN-ACK packet as a response, with its sequence number K and acknowledgment number J+1.

  • Step 3: Client responds with ACK

    • Upon receiving the SYN-ACK, the client sends an ACK packet with acknowledgment number K+1.
    • At this point, from the client's perspective, the connection is established (ESTABLISHED).

  • Step 4: Server processes ACK and transfers the connection

    • The server receives the client's ACK packet.
    • The server kernel removes the connection from the Half-Open Connection Queue.
    • The server kernel places the connection into the Established Connection Queue. The connection state on the server side also becomes ESTABLISHED.

  • Step 5: Application accepts the connection

    • The application on the server (e.g., Nginx, Apache) executes the accept() system call.
    • The accept() call retrieves (dequeues) an established connection from the head of the Established Connection Queue.
    • The application begins servicing the connection (e.g., processing an HTTP request).

3. Queue Overflow and Impact

Both queues have length limits. How new connection requests are handled when a queue is full is key to understanding queue-related issues.

  • Half-Open Queue Overflow

    • Cause: A large number of connection requests (SYN packets) arrive at the server in a short period, quickly filling the half-open queue.
    • Consequence: When the half-open queue is full, the server's default behavior upon receiving a new SYN packet is to drop the SYN packet, sending no response, or to reply with an RST (reset) packet to reject the connection. This causes the client to experience a connection timeout or failure.
    • Related Attack: The famous SYN Flood attack works by exhausting the server's half-open queue resources with SYN packets from spoofed source IPs, preventing legitimate users from connecting.

  • Established Queue Overflow

    • Cause: The server completes the three-way handshake, but the application's rate of calling accept() cannot keep up with the rate at which new connections enter the established queue. This may be due to application overload, blocking, or performance bottlenecks.
    • Consequence: When the established queue is full, how the server kernel handles newly completed connections (i.e., those just moved from the half-open queue) depends on the operating system configuration (e.g., the tcp_abort_on_overflow parameter):
      • Default behavior (typically): The kernel ignores the just-received ACK packet. That is, it pretends it didn't receive the ACK. The client believes the connection is established, but the server does not actually place the connection into the established queue. After a short delay, the server will retransmit the SYN-ACK packet (because it didn't receive the ACK, it assumes its SYN-ACK was lost).
      • Aggressive behavior: If configured to abort forcefully, the server may reply directly with an RST packet to reset the connection when the queue is full.

4. Related Configuration and Inspection

  • Half-Open Queue Size: In modern Linux, its length is typically determined by several parameters, including net.ipv4.tcp_max_syn_backlog (global maximum half-open connections), the backlog parameter of the application's listening port (set when calling listen(fd, backlog)), and is subject to system memory pressure.
  • Established Queue Size: Its length equals the smaller of the backlog parameter set in listen(fd, backlog) and the system-wide upper limit net.core.somaxconn (default value, e.g., 4096). For example, if backlog is set to 50 and somaxconn is 128, the maximum established queue length is 50.
  • Checking Queue Status: Linux commands such as netstat -s | grep -i "listen" or ss -lntp can be used to view the current Send-Q (maximum established queue length) for listening ports and statistics on queue overflow (e.g., times the listen queue of a socket overflowed).

Summary
The half-open connection queue and the established connection queue are core data structures in the TCP protocol stack implementation used to buffer connection requests. The half-open queue manages connections "in progress," while the established queue is the "waiting area" before connections are processed by the application. Queue overflow is a common cause of server connection issues (such as connection failures, timeouts). When optimizing server performance, it is necessary to adjust queue sizes reasonably based on actual load and ensure the application has sufficient capacity to promptly process established connections.