Pointer Walk

Caching Strategies

Cache-aside, read-through, write-through, write-behind and refresh-ahead: who writes, who reads, and when each serves stale data.

System DesignIntermediate12 min read

Two copies of the same data

A cache keeps a copy of data in memory so reads are fast. The moment there are two copies, the database and the cache, two questions appear:

  1. Who fills the cache? The app, or the cache itself?
  2. What happens on a write? Which copy is updated first, and when does the other one catch up?

A caching strategy is one set of answers to those questions. There are five common ones. They differ in who does the work, how fast reads and writes are, and when a reader can get an old answer. An old answer is called stale data.

Try it first

Pick a strategy and step through it. The boxes under the cache and the database show what each one holds. A red box is stale: it no longer matches the truth.

Cache-asideInteractive
Strategy
App
Cache
empty
Database
price = 10

Watch who talks to whom, and what the cache and the database hold after each message.

Cache-aside

The app does everything. It looks in the cache, and on a miss it reads the database and puts the answer in the cache itself. This is also called lazy loading, because nothing is cached until someone asks for it.

Cache-asideInteractive
App
Cache
empty
Database
price = 10

Watch who talks to whom, and what the cache and the database hold after each message.

python
def get_price(product_id):
    price = cache.get(product_id)
    if price is None:                      # miss
        price = database.read(product_id)
        cache.set(product_id, price, ttl=300)
    return price

def set_price(product_id, price):
    database.write(product_id, price)
    cache.delete(product_id)               # the next read will refill it

Good at

  • Works with any cache and any database
  • Only data that is actually read gets cached
  • If the cache goes down, the app can still read the database

Where it bites

  • The first read of anything is a miss
  • The caching logic is repeated wherever the app reads data
  • Stale between the database write and the cache delete, or for longer if the delete is missed

Read-through

The cache fills itself. The app only ever asks the cache. On a miss, the cache loads the value from the database, keeps it, and returns it.

It is the same flow as cache-aside with the loading moved out of the app and into the cache.

Read-throughInteractive
App
Cache
empty
Database
price = 10

Watch who talks to whom, and what the cache and the database hold after each message.

Good at

  • The app's code is simple: one place to read from
  • Loading logic lives in one place

Where it bites

  • The first read of anything is still a miss
  • Needs a cache that knows how to load from your database
  • Stale after a write until the copy expires, unless writes also go through the cache

Write-through

Writes go through the cache. The app writes to the cache, and the cache writes to the database before it says "done". Both copies change together.

Write-throughInteractive
App
Cache
price = 10
Database
price = 10

Watch who talks to whom, and what the cache and the database hold after each message.

It is usually paired with read-through, so that reads and writes both pass through the cache.

Good at

  • Reads after a write are fresh
  • The cache is already warm for data that was just written

Where it bites

  • Every write is slower: two hops instead of one
  • Caches data that may never be read
  • Stale if anything writes to the database without going through the cache

Write-behind

Write to the cache now, to the database later. The cache says "done" immediately and saves the change to the database in the background, often many changes in one batch. This is also called write-back.

Write-behindInteractive
App
Cache
price = 10
Database
price = 10

Watch who talks to whom, and what the cache and the database hold after each message.

Good at

  • The fastest writes of the five
  • Many small writes become a few large ones, which is gentle on the database

Where it bites

  • If the cache crashes before it saves, those writes are gone
  • The database is the stale copy, so anything reading it directly sees old data

Refresh-ahead

Reload popular items before they expire. Every cached copy has an expiry time. With refresh-ahead, a read that arrives shortly before that time is answered at once, and the cache reloads the value in the background.

Refresh-aheadInteractive
App
Cache
price = 10
Database
price = 12

Watch who talks to whom, and what the cache and the database hold after each message.

Good at

  • Popular items never cause a slow miss
  • No pile-up of requests when a popular item expires

Where it bites

  • The read that triggers the reload still gets the old value
  • Reloads items that nobody may ask for again

Side by side

StrategyWho fills the cacheWrite speedWhen a reader can get stale data
Cache-asideThe app, on a missNormalBetween the write and the delete, or until expiry if the delete is missed
Read-throughThe cache, on a missNormalAfter any write, until expiry
Write-throughThe cache, on every writeSlowerOnly if something bypasses the cache
Write-behindThe cache, on every writeFastestReaders of the database, until the cache saves
Refresh-aheadThe cache, before expiryNormalOnce, on the read that triggers the reload

Every row has an entry in the last column. No strategy removes stale data. Each one moves it somewhere you can live with.

Should you cache it at all?

A cache is a second copy to keep correct, so it should earn its place. Three questions decide it.

Should you cache it?Interactive

Is it read much more often than it changes?

A product page is read thousands of times per edit. A live counter changes on every request.

Key takeaways

  • A caching strategy answers two questions: who fills the cache, and what happens on a write.
  • Cache-aside: the app does the work. It is the most common and works anywhere.
  • Read-through and write-through move that work into the cache.
  • Write-behind makes writes instant and risks losing them.
  • Refresh-ahead reloads popular items before they expire.
  • Every strategy can serve stale data. Choose where and for how long.

Your progress

Check your progress

0%

  1. Read the page50% of this page

    When you have gone through the page and its animations, tick it off.

  2. Take the quiz50% of this page

    9 questions, from easy to hard. Take it as often as you like: the questions change, and only your best score counts.

Saved on this device only. See all your progress