Yes, you can just do:
mylock = multiprocessing.Lock()
with mylock:
...
as Lock is a context manager. So is RLock, and Lock and RLock from threading.
The documentation does state that it is "a clone of threading.Lock", so you can refer to "Using locks, conditions, and semaphores in the with statement"
[edit 2020: The documentation now mentions this explicitly]