Understanding Deadlocks in Python with Examples

Deadlocks are a common issue in concurrent programming, where multiple threads or processes are involved. They can cause your program to freeze indefinitely, making it unresponsive. This article will help you understand what deadlocks are, how they occur, and how to avoid them in Python.

What is a Deadlock?

A deadlock is a situation where two or more threads or processes are unable to proceed because each is waiting for the other to release a resource (Lock).

Imagine two bartenders working at a bar, each mixing a drink that requires the same two shakers. If both bartenders grab one shaker and wait for the other, they’ll never be able to finish their drinks, leading to a deadlock.

In programming, this happens when threads or processes hold onto resources and wait for others to release them, creating a cycle of dependency that halts progress.

Common Causes of Deadlocks

Deadlocks can occur in various ways, but here are some common scenarios:

1. Circular Wait
2. Self-Deadlock
3. Resource Starvation

Below I will show examples with code for better understanding.

Circular Wait

Like in example with two bartenders and shakers, circular wait can happen when thread A holds Lock 1 and waits for Lock 2, while Thread B holds Lock 2 and waits for Lock 1. Hopefully some example can make it even more clear.

In this code:

1. thread_a acquires lock1 and then attempts to acquire lock2.
2. thread_b acquires lock2 and then attempts to acquire lock1.
3. Both threads get stuck waiting for the lock held by the other thread, resulting in a circular wait deadlock.

As the result we only will see these logs before eternal hanging.

Thread A acquired Lock 1 
Thread B acquired Lock 2 
Thread A trying to acquire Lock 2 
Thread B trying to acquire Lock 1

Self-Deadlock Example

Such situation is rather rare but still useful to mention. During self-deadlock thread tries to acquire the same lock twice and as the result hanging forever.

In this example:

1. The thread acquires the lock the first time without any problems.
2. When it tries to acquire the same lock a second time, it gets stuck waiting for itself to release the lock, resulting in a deadlock.
3. The finally block and the program’s completion message will not execute because the thread is indefinitely blocked.

Resource Starvation

In this case a thread fails to release a resource (Lock), causing other threads to wait indefinitely. Usually is caused by loops or exceptions that are not handled properly.

In this code:

1. thread_with_exception acquires the lock and then raises an exception. The lock should ideally be released in the finally block, but if this is not done, the lock remains held.
2. waiting_thread attempts to acquire the lock. It will be blocked indefinitely because thread_with_exception does not release the lock due to the unhandled exception.

As the result we only see following logs before freezing forever. :(

Thread with Exception acquired the lock. 
Exception caught: An unexpected error occurred! 
Waiting Thread trying to acquire the lock...

How to Avoid Deadlocks

Here are some best practices to avoid deadlocks:

1. Use Context Managers: Use with statement to manage locks. In this case locks will be released properly, even if an error occurs.

2. Consistent Lock Ordering: Make sure that you always acquire locks in the same order. This prevents circular wait conditions.

3. Timeouts: Use timeout parameter when acquiring locks to avoid waiting indefinitely. Such simple trick can save from many unpredictable deadlocks in codebase with complicated logic.

4. Avoid Nested Locks: Try to minimise the use of nested locks. If you can, design your program so that it doesn’t need to hold multiple locks simultaneously. Debugging code with lots of locks can be incredibly challenging and frustrating.

5. Keep Lock Scope Small: Hold locks for the shortest time possible. Perform non-critical operations outside the critical section to shorten the duration locks are held.

6. Deadlock Detection and Recovery: Implement deadlock detection mechanisms to monitor and recover from deadlocked states. This can be complex but useful for critical applications.

7. Concurrency Libraries: Use higher-level concurrency libraries like concurrent.futures which abstract away much of the complexity of threading and locking.

Conclusion

Deadlocks can be tricky to debug and resolve, but understanding their causes and following best practices can help you avoid them. I hope now you have a better understanding of deadlocks and will avoid them in your code!