Pointer Walk

Race Conditions

Two clicks, and one like goes missing. Every line is correct; the gap between reading and writing is the bug.

ConcurrencyBeginner8 min read

Two clicks, one like

A post has 41 likes. Two people press Like at the same moment. The counter now says 42.

One like has gone missing, and there is no error anywhere. Here is the code that handled each click:

python
def like(post_id):
    likes = db.read_likes(post_id)        # 1. read
    likes = likes + 1                     # 2. add one
    db.write_likes(post_id, likes)        # 3. write

Read it as many times as you like. Every line is correct. Run it once and it works. The bug is not in any line. It is in the gap between line 1 and line 3.

What went wrong

Two clicks at the same momentInteractive
Click 1
Database
likes = 41
Click 2

A post has 41 likes. Two people press Like at almost the same time.

  1. Click 1 reads 41

    It now holds a copy of the number.

  2. Click 2 also reads 41

    Click 1 has not written yet, so the database still says 41.

  3. Click 1 writes 42

    Correct, as far as it knows.

  4. Click 2 writes 42

    Also "correct": 41 + 1. But its 41 was out of date. It overwrites Click 1's result with the same number.

This is a race condition: the outcome depends on the exact order in which two things happen, and at least one possible order gives a wrong result. This particular kind is called a lost update.

Why it is so hard to find

Now run the same code with slightly different timing.

The same code, with better timingInteractive
Click 1
Database
likes = 41
Click 2

A post has 41 likes. Two people press Like at almost the same time.

It works. The window for the bug is the few milliseconds between a read and its write. Two requests have to land inside that window together.

  • In development, you click once. It never happens.
  • In testing, requests come one after another. It never happens.
  • In production, with thousands of requests a second, it happens a few times a day.

And when it does, nothing crashes and nothing is logged. A counter is slightly off. Stock goes below zero. A balance is wrong by one payment. You find out weeks later, and cannot reproduce it.

The pattern to look for

Any time code does this, it can race:

  1. Read a shared value,
  2. decide or calculate something from it,
  3. write based on what was read.

If anyone else can change the value between step 1 and step 3, the decision was made on old information. This pattern is called read-modify-write, or check-then-act.

Example

The same bug in other clothes

Selling the last ticket. Read "1 left". Check it is more than 0. Sell. Two buyers both read 1, both pass the check, both buy.

Creating a username. Check "sam" is not taken. Insert it. Two people both check, both find it free, both insert.

Withdrawing money. Read the balance of 100. Check 100 is at least 80. Subtract 80. Two withdrawals of 80 both pass the check.

Fix 1: make it one step

The cleanest fix is to remove the gap. Ask the database to do the reading and the writing itself, as a single atomic operation: one that either happens completely or not at all, with nothing able to slip into the middle.

sql
UPDATE posts SET likes = likes + 1 WHERE id = 7;
The fix: read and write in one stepInteractive
Click 1
Database
likes = 41
Click 2

A post has 41 likes. Two people press Like at almost the same time.

The program never holds a stale copy, because it never holds a copy at all.

The same idea fixes the other examples:

sql
-- Sell a ticket only if one is left. Check how many rows were changed.
UPDATE events SET tickets = tickets - 1 WHERE id = 3 AND tickets > 0;

For the username, put a unique constraint on the column and just insert. The database refuses the second one.

Fix 2: take a lock

When the logic is too complicated for one statement, make everyone else wait while you do your read-modify-write. A lock guarantees only one piece of code is in that section at a time.

python
with lock:
    likes = read_likes(post_id)
    write_likes(post_id, likes + 1)

In a database, SELECT ... FOR UPDATE locks the rows you read until your transaction ends. Locks work, at the cost of making others wait, and with a risk of deadlocks if several are taken in different orders.

Fix 3: notice, and try again

Let both proceed, but detect the collision at the moment of writing. Store a version number with the row, and write only if the version is still the one you read.

sql
UPDATE posts SET likes = 42, version = 6
WHERE id = 7 AND version = 5;

If someone else got there first, the version is no longer 5, no row is updated, and you know to read again and retry. This is called optimistic locking: assume there will be no conflict, and check.

Two clicks at the same momentInteractive
Scenario
Click 1
Database
likes = 41
Click 2

A post has 41 likes. Two people press Like at almost the same time.

It is not only databases

The same bug happens inside a single program whenever two threads share a variable. count += 1 looks like one step. It is three: read the value, add one, store it. Two threads can interleave exactly as the two clicks did.

The fixes are the same: an atomic operation, a lock, or not sharing the variable at all.

Key takeaways

  • A race condition is a bug that depends on the order in which concurrent operations happen.
  • It appears wherever code reads a value, decides something, and writes back, with a gap in between.
  • Every line can be correct. The gap is the bug.
  • It is rare, silent and hard to reproduce, so it survives testing.
  • Fix it with an atomic operation, a lock, or a version check with retry.

Your progress

Check your progress

0%

  1. Read the page50% of this page

    When you have gone through the page and its animations, tick it off.

  2. Take the quiz50% of this page

    7 questions, from easy to hard. Take it as often as you like: the questions change, and only your best score counts.

Saved on this device only. See all your progress