parallelizing prime generation in sage

Question

Since ask.sagemath.org is down, I figure I'd ask this here...

I'm trying to parallelize the generation of a bunch of random primes using the random_prime function in sage. Here's some code:

#!/usr/bin/env sage -python
from __future__ import print_function
from sage.all import *
import time

N = 100
B = 200
length = (1 << B) - 1

print('Generating primes (sequentially)')
start = time.time()
ps = []
for _ in xrange(N):
    ps.append(random_prime(length))
end = time.time()
print('Took: %f seconds' % (end - start))
print(ps)

@parallel(ncpus=10)
def _random_prime(length):
    return random_prime(length)

print('Generating primes (in parallel)')
start = time.time()
ps = [length] * N
ps = list(_random_prime(ps))
end = time.time()
print('Took: %f seconds' % (end - start))
ps = [p for _, p in ps]
print(ps)

The first run through computes the primes sequentially, and it works.

The second run through computes them using sage's @parallel decorator. It "works" in the sense that the computation is parallelized, but all the output primes are the same (i.e., it doesn't generate 100 random primes but rather 100 instances of the same random prime). I'd think this'd be a common usecase of @parallel, however I cannot find any details on what the issue here is. Anyone have any ideas? Thanks.

Answer 1

Just a hint on this - is it possible that you'd need to set a different seed each time? Note that the doc doesn't have a #random in the test, so perhaps the seed for all the parallel instances is the same for some reason (which seems reasonable).

Edit to put Volker's more detailed description of how to do this:

import os
import sage.misc.randstate as randstate
randstate.set_random_seed(os.getpid())