Is Python cStringIO thread-safe?

Question

As title say, does Python cStringIO protect their internal structures for multithreading use?

Thank you.

Answer 1

Take a look at an excellent work on explaining GIL, then note that cStringIO is written purely in C, and its calls don't release GIL.

It means that the running thread won't voluntarily switch during read()/write() (with current virtual machine implementation). (The OS will preempt the thread, however other Python threads won't be able to acquire GIL.)

Taking a look at the source: Python-2.7.1/Modules/cStringIO.c there is no mention about internals protection. When in doubt, look at source :)

Answer 2

I assume you are talking about the CPython implementation of Python.

In CPython there is a global interpreter lock which means that only a single thread of Python code can execute at a time. Code written in C will therefore also be effectively single threaded unless it explicitly releases the global lock.

What that means is that if you have multiple Python threads all using cStringIO simultaneously there won't be any problem as only a single call to a cStringIO method can be active at a time and cStringIO never releases the lock. However if you call it directly from C code which is running outside the locked environment you will have problems. Also if you do anything more complex than just reading or writing you will have issues, e.g. if you start using seek as your calls may overlap in unexpected ways.

Also note that some methods such as writelines can invoke Python code from inside the method so in that case you might get other output interleaved inside a single call to writelines.

That is true for most of the standard Python objects: you can safely use objects from multiple threads as the individual operations won't break, but the order in which things happen won't be defined.

Answer 3

No it is not currently thread safe.

Answer 4

It is as "thread-safe", as file operations can be (which means — not much). The Python implementation you're using has Global Interpreter Lock (GIL), which will guarantee that each individual file operation on cStringIO will not be interrupted by another thread. That does not however guarantee, that concurrent file operations from multiple threads won't be interleaved.