Python Cookbook

Allowing Multithreaded Read Access While Maintaining a Write Lock

Credit: Sami Hangaslammi

Problem

You need to allow unlimited read access to a resource when it is not being modified while keeping write access exclusive.

Solution

“One-writer, many-readers” locks are a frequent necessity, and Python does not supply them directly. As usual, they’re not hard to program yourself, in terms of other synchronization primitives that Python does supply:

import threading

class ReadWriteLock:
    """ A lock object that allows many simultaneous "read locks", but
    only one "write lock." """

    def _ _init_ _(self):
        self._read_ready = threading.Condition(threading.Lock(  ))
        self._readers = 0

    def acquire_read(self):
        """ Acquire a read lock. Blocks only if a thread has
        acquired the write lock. """
        self._read_ready.acquire(  )
        try:
            self._readers += 1
        finally:
            self._read_ready.release(  )

    def release_read(self):
        """ Release a read lock. """
        self._read_ready.acquire(  )
        try:
            self._readers -= 1
            if not self._readers:
                self._read_ready.notifyAll(  )
        finally:
            self._read_ready.release(  )

    def acquire_write(self):
        """ Acquire a write lock. Blocks until there are no
        acquired read or write locks. """
        self._read_ready.acquire(  )
        while self._readers > 0:
            self._read_ready.wait(  )

    def release_write(self):
        """ Release a write lock. """
        self._read_ready.release(  )

Discussion

It is often convenient to allow unlimited read access to a resource when it is not being modified and still keep write access exclusive. While the threading module does not contain a specific class for the job, the idiom is easy to implement using a Condition object, and this recipe shows how you can do that.

An instance of the ReadWriteLock class is initialized without arguments, as in:

rw = ReadWriteLock(  )

Internally, rw._readers counts the number of readers who are currently in the read-write lock (initially zero). The actual synchronization is performed by a threading.Condition object (created at _ _init_ _ around a new Lock object and held in rw._read_ready).

The acquire_read and release_read methods increment and decrement the number of active readers. Of course, this happens between acquire and release calls to _read_ready—such bracketing is obviously necessary even to avoid race conditions between different threads wanting to acquire or release a read lock. But we also exploit _read_ready for another purpose, which is why release_read also does a notifyAll on it, if and when it notices it has removed the last read lock.

The notifyAll method of a Condition object wakes up all threads (if any) that are on a wait condition on the object. In this recipe, the only way a thread can get into such a wait is via the acquire_write method, when it finds there are readers active after acquiring _read_ready. The wait call on the Condition object releases the underlying lock, so release_read methods can execute, but reacquires it again before waking up, so acquire_write can safely keep checking whenever it wakes up, if it’s finally in a no-readers-active situation. When that happens, acquire_write returns to its caller, but keeps the lock, so no other writer or reader can enter again, until the writer calls release_write, which lets the lock go again.

Note that this recipe offers no guarantee against what is technically known as a starvation situation. In other words, there is no guarantee that a writer won’t be kept waiting indefinitely by a steady stream of readers arriving, even if no reader keeps its read lock for very long. If this is a problem in your specific application, you can avoid starvation by adding complications to ensure that new readers don’t enter their lock if they notice that a writer is waiting. However, in many cases, you can count on situations in which no readers are holding read locks, without special precautions to ensure that such situations occur. In such cases, this recipe is directly applicable, and besides eschewing complications, it avoids potentially penalizing reader performance by making several readers wait for one pending writer.

See Also

Documentation of the standard library module threading in the Library Reference.