Redis Persistence Mechanisms and Performance Trade-offs

Redis Persistence Mechanisms and Performance Trade-offs

Topic Description: Please explain in detail the working principles of Redis's two persistence mechanisms (RDB and AOF), analyze their respective performance characteristics, data security, and recovery efficiency, and explain how to choose and configure optimizations based on business requirements in actual production environments.

Knowledge Explanation:

一、 The Necessity of Persistence
Redis data is stored in memory, offering extremely fast read and write speeds. However, memory is volatile; if the server restarts or crashes, all data will be lost. Persistence mechanisms save data from memory to the hard disk in some format, ensuring data is not lost due to process termination, thereby achieving permanent data storage.

二、 RDB Persistence Mechanism

  1. Core Concept: At specified time intervals, a snapshot of the dataset in memory is written to a binary file (default name dump.rdb).
  2. Trigger Methods:
    • Manual Trigger:
      • SAVE command: Blocks the current Redis server until the RDB process is complete. Prohibited in production environments as it causes prolonged service unavailability.
      • BGSAVE command: The Redis process forks a child process responsible for creating the RDB file, while the parent process (main process) continues handling client requests. This is the recommended trigger method.
    • Automatic Trigger: Configure save <seconds> <changes> rules in the Redis configuration file (redis.conf). For example:
      • save 900 1: Triggers BGSAVE if at least 1 key changes within 900 seconds (15 minutes).
      • save 300 10: Triggers BGSAVE if at least 10 keys change within 300 seconds (5 minutes).
      • save 60 10000: Triggers BGSAVE if at least 10000 keys change within 60 seconds.
  3. Workflow (BGSAVE):
    • The main process receives the BGSAVE command or meets the automatic trigger condition.
    • The main process calls the fork() system call to create a child process. At this point, the main and child processes share the same memory data.
    • The child process starts writing the shared memory data to a temporary RDB file.
    • After writing is complete, the child process atomically replaces the old RDB file with the new temporary file.
  4. Advantages:
    • High Performance: RDB is a data snapshot, ideal for backups, disaster recovery, and full replication (e.g., master-slave synchronization). Faster for restoring large datasets compared to AOF.
    • Compact Files: RDB files are compressed binary files, occupying less disk space.
  5. Disadvantages:
    • Lower Data Security: RDB takes periodic snapshots. If the server crashes between two snapshots, all data after the last snapshot is lost.
    • fork May Block Service: Although BGSAVE is non-blocking, the fork operation itself can be time-consuming with huge datasets, causing brief pauses in the main process (depending on system configuration and hardware).

三、 AOF Persistence Mechanism

  1. Core Concept: Records all write commands executed by Redis into a log file (similar to MySQL's binlog). When Redis restarts, it replays all commands in the AOF file to rebuild the data.
  2. Enabling and Sync Strategies: Enable by configuring appendonly yes in redis.conf. The sync strategy is controlled by the appendfsync option, key to performance and data security.
    • appendfsync always: Synchronously writes every write command to disk. Most secure, absolutely no loss of acknowledged writes, but poorest performance as disk I/O becomes the bottleneck.
    • appendfsync everysec: Synchronizes once per second. A compromise, at most 1 second of data loss. Good performance, the default recommended strategy.
    • appendfsync no: Lets the operating system decide when to sync. Best performance, but highest risk of data loss (typically up to 30 seconds of recent data).
  3. AOF Rewrite Mechanism:
    • Problem: The AOF file continuously appends commands, growing larger. Replaying all commands for recovery is slow.
    • Solution: AOF rewrite. Creates a new AOF file containing the minimum set of commands needed to rebuild the current dataset (e.g., 100 incr operations on a key rewritten as one set command).
    • Trigger Methods: Manual (BGREWRITEAOF command) or automatic (configure auto-aof-rewrite-percentage and auto-aof-rewrite-min-size).
    • Workflow (BGREWRITEAOF): Similar to BGSAVE, uses a forked child process, not blocking the main process.
  4. Advantages:
    • High Data Security: Depending on the sync strategy, at most 1 second of data is lost, or even none.
    • Good Readability: AOF files are plain text, viewable and modifiable manually (not recommended).
  5. Disadvantages:
    • Larger File Size: AOF files are typically larger than RDB files for the same dataset.
    • Slower Recovery Speed: Replaying all commands is slower than loading an RDB snapshot for large datasets.

四、 Performance Trade-offs and Production Practices

  1. Selection Strategy:

    • Pursue Extreme Performance, Tolerate Minute-level Data Loss: Such as caching scenarios, use RDB only.
    • Pursue Data Security, Cannot Tolerate Significant Data Loss: Such as session caching, persistent counters, prioritize AOF (configure appendfsync everysec).
    • Need Disaster Recovery and Fast Restart Speed: Enable both RDB and AOF. This is the most common production configuration. Use RDB for fast cold backups and AOF for data security. On restart, Redis prioritizes loading the AOF file for recovery as it's usually more complete.

  2. Configuration Optimization:

    • RDB Optimization: Adjust save rules to avoid overly frequent BGSAVE. For example, in businesses with infrequent data changes, intervals can be extended.
    • AOF Optimization: Ensure appendfsync everysec is used. Monitor AOF file size and trigger rewrites promptly. Set AOF rewrite triggers more leniently (e.g., auto-aof-rewrite-percentage 100, auto-aof-rewrite-min-size 64mb) to avoid frequent rewrites.
    • OS Optimization: Ensure sufficient system memory to reduce fork operation latency. Use fast disks (e.g., SSD) for persistence files.

Summary: RDB and AOF are the two pillars of Redis persistence. They are not opposing but complementary. Understanding their internal principles and performance impacts allows for the most reasonable configuration choices based on business requirements for data security and service performance.