Synchronize access to shell variables while using Gnu parallel (i.e. critical section in Gnu parallel)

StackOverflow https://stackoverflow.com/questions/23230969

Question

I have a for loop that I'd like to convert to parallel. However, I'm appending to a global (bash) array within my loop. What's the recommended way of dealing with this situation? Does parallel provide any form of synchronization between jobs besides --keep-order?

I thought about replacing my array with a combination of flock and echo > some_shared_file, but wanted to know if there's a standard way of implementing a "critical section" with parallel.

Was it helpful?

Solution

Bash only has "thread local" variables. There are no global variables that can be updated from different threads. All variables/arrays are copied to the subprocesses, and changes in one will not be reflected in another.

The more general answer -- if you had resources that actually could be updated from different processes -- would be to use sem which comes with GNU parallel.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top