TCP SYN Flood Attack and Protection Mechanisms

TCP SYN Flood Attack and Protection Mechanisms

Description
SYN Flood is a classic DDoS attack that exploits vulnerabilities in the TCP three-way handshake to exhaust server resources. Attackers send a large number of SYN packets with forged source IP addresses without completing the handshake, causing the server's half-open connection queue to fill up and preventing it from responding to legitimate requests.

Attack Principle Analysis

  1. Vulnerability in Normal Three-Way Handshake:

    • After the client sends a SYN, the server replies with a SYN-ACK and enters the SYN_RCVD state, waiting for the client's ACK
    • This connection is called a "half-open connection" and is stored in the half-open connection queue
    • If the client does not reply with an ACK, the server will retransmit the SYN-ACK until timeout (usually 1-3 minutes)

  2. Attacker's Exploitation Method:

    • Forge a large number of SYN packets with non-existent or unreachable source IP addresses
    • The server allocates connection resources for each SYN but never receives an ACK reply
    • Once the half-open connection queue is full, new connections cannot be established

Attack Process Demonstration
Step 1: The attacker forges a source IP (e.g., 192.0.2.1) and sends a SYN packet to the server
Step 2: The server responds with a SYN-ACK to the forged IP and reserves connection resources
Step 3: The real host 192.0.2.1 receives the unexpected packet and sends an RST or discards it
Step 4: The server continuously retransmits SYN-ACK until timeout, during which resources remain occupied

Protection Mechanisms Detailed Explanation

  1. SYN Cookie Technology (Operating System Level)

    • Working Principle:
      • Do not allocate connection resources immediately upon receiving a SYN
      • Calculate a hash value based on connection information (five-tuple + timestamp) as the initial sequence number
      • Send the hash value as the sequence number in the SYN-ACK to the client
    • Verification Process:
      • When an ACK is received, check whether the acknowledgment number minus 1 matches the locally calculated hash value
      • Allocate full connection resources only after verification passes
    • Advantages: Completely prevents half-open connection queue overflow
    • Disadvantages: Hash calculation increases CPU overhead and does not support TCP option negotiation

  2. Connection Limit (Firewall Level)

    • Set thresholds for half-open connections per IP (e.g., 10 per second)
    • Discard new SYN packets from that IP if the threshold is exceeded
    • Combine with an allowlist mechanism to avoid blocking legitimate users

  3. First Packet Discard Strategy (Load Balancer Level)

    • Intentionally discard the first SYN packet and wait for the client to retransmit
    • A real TCP stack will retransmit quickly, while forged tools usually do not retransmit
    • Distinguish between legitimate users and attack packets based on retransmission behavior

  4. TCP Proxy Mode (Cloud Protection Solution)

    • Deploy a proxy device in front of the server to complete the three-way handshake
    • Only forward fully established connections to the backend server
    • Completely isolate the impact of half-open connections on the server

Practical Deployment Recommendations

  • Enable SYN Cookie on Linux: echo 1 > /proc/sys/net/ipv4/tcp_syncookies
  • Adjust half-open connection queue size: sysctl -w net.ipv4.tcp_max_syn_backlog=2048
  • Reduce SYN-ACK retry time: sysctl -w net.ipv4.tcp_synack_retries=3
  • Combine multi-layer protection: Perimeter firewall + Load balancer + Operating system parameter optimization

By combining the above mechanisms, SYN Flood attacks can be effectively mitigated while ensuring normal service.