What is HTTP caching#
Why do we need caching?
We know that every time we visit a webpage, we need to load resources. If these resources are loaded from a remote server, the entire request and response time will be long. Is there a faster way? Yes, we can request resources from a nearby CDN, reducing long-distance network transmission and naturally improving response speed. However, the CDN is still a bit far from the client. Is there a way to cache data in the browser and minimize external resource requests? In fact, these ideas are all about caching. Caching at each node can greatly reduce remote resource requests and speed up response time.
There are several concepts in the cache
- Stale Content: Outdated content, that is, cached and expired content
- Fresh Content: Fresh content, that is, cached and not expired content
- Cache Validation: It is the process of contacting the server to check the validity of the cached content and getting updated when it is about to expire.
- Cache Invalidation: It is the process of removing any stale content available in the cache.
Caches can be divided into browser cache, proxy cache, and reverse proxy cache based on location.
Browser Cache#
Browser cache is limited to a single user and is different from other caches, and it can store private responses.
Proxy Cache#
Unlike the browser cache that serves a single user, the proxy cache can serve hundreds of different users accessing the same content. It is often implemented by ISPs.
Reverse Proxy Cache#
Reverse proxy cache is generally deployed on the server side to reduce the load on the server.
Cache Headers#
Whenever a server sends a response, it is accompanied by some HTTP headers to guide caching and how to cache this response.
Expires#
Before HTTP/1.1 and the introduction of cache control, there was an Expires header, which was just a timestamp that told the cache how long certain content should be considered fresh. For example,
Expires: Mon, 13 Mar 2017 12:22:00 GMT
Of course, the clock on the cache must be synchronized with the clock on the server, otherwise the desired results may not be achieved.
Although Expires is still in use, we should now prioritize Cache-Control.
Pragma#
This is also an old feature from HTTP/1.1. We may see Pragma: no-cache used here and there, hoping to prevent the response from being cached.
Cache-Control#
Cache-Control specifies how long content should be cached and how it should be cached.
Cache-Control can contain multiple values,
private
Setting the cache to private means that the content will not be cached in any proxy, it will only be cached by the client (i.e., the browser).
public
If set to public, it can be cached by proxy agencies in addition to being cached by the client; it serves many other users.
no-store
Specifies that the content should not be cached by any cache.
no-cache
no-cache means that the cache can be kept, but the cached content needs to be revalidated from the server before it is served (e.g., using ETag). In other words, there is still a request to the server, but it is for validation, not for downloading the cached content.
max-age: seconds
max-age specifies the number of seconds the content should be cached. For example, if the cache-control looks like this:
cache-control: max-age=900, private
s-maxage: seconds
s-maxage, where the s-prefix stands for shared. This directive is specifically for shared caches. Like max-age, it also specifies the number of seconds something should be cached. If present, it overrides the max-age and expires headers for shared caches.
must-revalidate
Sometimes, if you have network issues and the content cannot be retrieved from the server, the browser may serve stale content without validation. must-revalidate can prevent this situation. If this directive is present, it means that stale content cannot be served under any circumstances and must be revalidated from the server before providing the data.
proxy-revalidate
proxy-revalidate is similar to must-revalidate, but it applies to shared or proxy caches. In other words, proxy-revalidate is to must-revalidate as s-maxage is to max-age.
Mixing Values
We can combine these directives in different ways to achieve different caching behaviors, but no-cache/no-store and public/private are mutually exclusive. If no-store and no-cache are specified together, no-store takes precedence over no-cache.
Validators#
So far, we have only discussed how content is cached and how long cached content is considered fresh, but we have not discussed how the client validates against the server. Let's now discuss the headers used for this purpose.
Etag
Etag, or "Entity Tag," was introduced in the HTTP/1.1 specification. Etag is simply a unique identifier that the server associates with some resource. This ETag is later used by the client to make conditional HTTP requests, saying "give me this resource if the ETag doesn't match mine," and the content is only downloaded if the ETag doesn't match.
The method of generating Etag is not specified in the HTTP documentation, but typically a collision-resistant hash function is used to assign Etags to each version of a resource. There can be two types of etags, strong and weak.
ETag: "j82j8232ha7sdh0q2882" - Strong Etag
ETag: W/"j82j8232ha7sdh0q2882" - Weak Etag (prefixed with `W/`)
A strong Etag means that two resources are exactly the same, there is no difference between them. A weak Etag means that although two resources are not exactly the same in the strict sense, they can be considered the same. For example, a weak Etag may be useful for dynamic content.
Now that we know what an Etag is, how does the browser make this request? By making a request to the server and sending the available Etag in the If-None-Match header.
Consider the following scenario: you open a webpage that loads a logo image with a cache period of 60 seconds and an Etag of abc123xyz. About 30 minutes later, you reload the webpage and the browser notices that the fresh logo that was valid for 60 seconds is now stale; it triggers a request to the server, sending the Etag of the stale logo image in the if-none-match header.
If-None-Match: "abc123xyz"
The server then compares this Etag with the current version of the resource. If both Etags match, the server sends back a 304 Not Modified response, which tells the client that the copy it has is still good and it will be considered fresh for another 60 seconds. If the two Etags don't match, meaning the Etag has likely changed, the client is sent the new Etag, which it will use to replace the stale one it has.
Last-Modified
The server may include the Last-Modified header, indicating the date and time when some content was last modified.
Last-Modified: Wed, 15 Mar 2017 12:30:26 GMT
When the content expires, the client makes a conditional request to the server, including the last modification date in the If-Modified-Since header, to get an updated Last-Modified date; if it matches the date on the client, the Last-Modified date of the content will be updated to be considered fresh for another n seconds. If the received Last-Modified date does not match the date on the client, the content must be reloaded from the server and replaced with the stale content the client has.
If-Modified-Since: Wed, 15 Mar 2017 12:30:26 GMT
Now you may wonder, what if the cached content has both Last-Modified and ETag? In that case, both are used, meaning the content will not be re-downloaded only if the Etag matches the newly retrieved content and the Last-Modified date also matches. If the Etag doesn't match or the Last-Modified date is greater than the date on the server, the content must be re-downloaded.