python locking and threading concurrency [duplicate]

Question

This question already has answers here:

Threading Best Practices (11 answers)

Closed 8 years ago.

i have a question regards python locks and threading, i realise locks are used to prevent variables from being overwritten by another thread, is it normal to use locks to get around this issue as it then means you can only run a single thread concurrently, it also means creating acquire/release locks for every variable that maybe overwritten, which for my project runs into quite a few!

how are people doing this?, wrapping the variables in thread safe lists or creating unique variables based on the thread name perhaps?, or is everybody littering their code with lock acquire and releases?.

Answer 1

The best idea is to just not use threads at all. Most Python implementations have a global interpreter lock which eliminates the advantages of using threads in first place. If you're using threading to wait for IO, you can get the same or better performance if you just use asynchronous IO instead. If you're using threading to perform computation (number crunching) across processors, the python global lock prevents it from working so you're better using multiple processes instead.

In contrast with having no upside, threading in python have lots of shortcomings and caveats, like you already found out. You still have to do data sharing control, and to deal with oddities related to threads receiving cpu attention in moments you don't control. All that for no benefit.

TL;DR just don't use threads