Detailed Explanation of HTTP Caching Mechanism

Detailed Explanation of HTTP Caching Mechanism

Problem Description: Please explain the HTTP caching mechanism in detail, including its core concepts, how different types of caching strategies (strong caching and negotiated caching) work, relevant HTTP header fields, and the complete cache validation process.

Knowledge Explanation:

HTTP caching is a technique that stores copies of requested resources and reuses those copies for subsequent requests instead of fetching them again from the origin server. It can significantly reduce network latency and bandwidth consumption, improving website performance and user experience. Its core process can be divided into two steps: Cache Storage (whether to store a copy) and Cache Validation (whether the copy is still valid).

Step One: Core Concept - Cache Location

Requests made by the browser may have their caches stored in the following locations, listed in order of priority from high to low:

  1. Service Worker Cache: A programmatically controlled cache managed precisely by JavaScript scripts.
  2. Memory Cache: Memory cache, extremely fast to read, but its lifecycle is tied to the browser tab.
  3. Disk Cache: Disk cache, large capacity, long persistence, and the primary carrier of HTTP caching.
  4. Network Request: When all caches miss, a request is made to the server.

Our discussion focuses mainly on the Disk Cache controlled by HTTP headers.

Step Two: Cache Classification - Strong Caching

The core idea of strong caching is: during the cache's validity period, the client does not communicate with the server and directly uses the local cache. Its validity is determined by one of the following two HTTP header fields:

1. Cache-Control (HTTP/1.1, higher priority)
This is a general header that controls caching behavior through directives. Common directives include:

  • max-age=: This is the most commonly used directive. For example, Cache-Control: max-age=3600 indicates that the resource can be cached on the client for 3600 seconds (1 hour) from the time it was requested. During this period, subsequent accesses to the same resource will hit the strong cache.
  • no-cache: This name is somewhat misleading. It does not mean "do not cache"; rather, it means "do not use strong caching." When this directive is used, the client must validate with the server whether the cache is fresh before each request (i.e., it proceeds to the "negotiated caching" process described later).
  • no-store: This is the true "do not cache." It indicates that the client should not store any part of the request or response and must fetch it entirely from the server each time.
  • public: The response can be cached by any intermediate node (e.g., a proxy server).
  • private: The response can only be cached by a single user's browser, typically used for private data.

2. Expires (HTTP/1.0)
This is an absolute timestamp. For example, Expires: Wed, 21 Oct 2024 07:28:00 GMT. It indicates the expiration time of the resource. The drawback is that if the client's and server's clocks are not synchronized, it can lead to incorrect cache decisions. In modern networks, max-age (a relative time) from Cache-Control is the preferred choice.

Strong Caching Process:

  1. The browser requests a resource for the first time. The server returns the resource along with response headers like Cache-Control or Expires.
  2. The browser caches the resource along with these response headers.
  3. When requesting the same resource again, the browser first checks the cache.
  4. Determines if the cache is still valid (by calculating max-age or comparing with the Expires time):
    • If valid: Then a strong cache hit occurs. The browser loads the resource directly from local disk or memory. The HTTP status code is 200 (from disk cache) or 200 (from memory cache), and in the network request panel, the request appears grayed out, with clear indication in the Size column.
    • If expired: Strong caching fails, and the process moves to the next step—negotiated caching.

Step Three: Cache Classification - Negotiated Caching

Negotiated caching, also known as conditional caching. When strong caching fails, the browser must ask the server with some "identifier": "Is the version I cached earlier still usable?" The server uses these identifiers to determine if the cache is still valid.

Negotiated caching involves two pairs of header fields:

1. Last-Modified / If-Modified-Since

  • Workflow:

    • First request: The server returns Last-Modified: GMT in the response headers, indicating the last modification time of the resource.
    • Subsequent request: The browser includes If-Modified-Since: GMT in the request headers. This value is the Last-Modified value received earlier.
    • Server validation: The server compares the resource's current last modification time with the time from If-Modified-Since.
      • If the times are the same (resource not modified), it returns 304 Not Modified with an empty response body. Upon receiving 304, the browser loads the resource from its cache.
      • If the times differ (resource modified), it returns 200 OK with the full resource content.

  • Drawbacks:

    • Precision is limited to seconds. Changes within one second cannot be detected.
    • A file might be regenerated without actual content changes, but the modification time updates, leading to unnecessary transfers.

2. ETag / If-None-Match (higher priority)
To address the shortcomings of Last-Modified, HTTP/1.1 introduced ETag.

  • Workflow:

    • First request: The server generates a unique identifier for the resource (often a hash value) and returns it in the response header as ETag: "xyz123".
    • Subsequent request: The browser includes If-None-Match: "xyz123" in the request headers. This value is the ETag value received earlier.
    • Server validation: The server calculates the current resource's ETag and compares it with the value from If-None-Match.
      • If the values are the same (resource not modified), it returns 304 Not Modified.
      • If the values differ (resource modified), it returns 200 OK with the full resource content.

  • Advantage: ETag offers higher precision and can accurately detect changes in resource content.

Step Four: Complete Cache Decision Flow

Let's connect the above steps to form a complete flowchart:

  1. Initiate Request: The browser prepares to send a GET request to fetch a resource (e.g., an image file).
  2. Check Strong Cache:
    • Check if a local cache for this resource exists.
    • If yes, examine its Cache-Control's max-age or Expires value to determine if it's still valid.
    • If strong cache is valid: Use the cache directly, request ends.
  3. Strong Cache Expired, Prepare for Negotiated Cache:
    • The browser builds a request that will carry the cache's "identifier(s)".
    • If the cache has an ETag, add If-None-Match to the request headers.
    • If the cache has a Last-Modified, add If-Modified-Since to the request headers.
  4. Send Request to Server.
  5. Server Validation:
    • The server first checks If-None-Match to see if the ETag matches.
    • If the ETag matches, return 304 Not Modified.
    • If the ETag does not match or is not provided, fall back to using If-Modified-Since to check the modification time.
    • If the times are consistent, return 304 Not Modified.
    • If neither matches, the resource has been updated, so return 200 OK with the latest resource content.
  6. Browser Receives Response:
    • If 304 is received: The browser loads the resource from its cache.
    • If 200 is received: The browser uses the new resource and updates its local cache based on the new response headers.

Summary: HTTP caching is a hierarchical decision system controlled by request/response headers. Strong caching (no request sent) takes precedence over negotiated caching (validation request sent). By properly configuring Cache-Control (e.g., max-age) and ETag, an optimal balance can be achieved between performance (reducing requests) and accuracy (obtaining the latest content).