Question

I need to construct the cartesian product of output generated by two or more generator functions on the fly.
I would like for itertools.product not to block, but instead to give mr the first product elements before the generator input functions raised StopIteration.
Is there a function that provides sth like that?

I wrote a simple program to demonstrate my problem:

#!/usr/bin/python
import time
import itertools

def g_delay(l, delay):
    for i in range(l):
        yield i
        time.sleep(delay)

def g(l):
    for i in range(l):
        yield i

if __name__ == "__main__":
    start_time = time.time()
    p = itertools.product(g_delay(2,1), g_delay(3,1))
    elapsed_time = time.time() - start_time
    print '%f' % elapsed_time
    for i in p:
        print i

    print

    start_time = time.time()
    p = itertools.product(g(2), g(3))
    elapsed_time = time.time() - start_time
    print '%f' % elapsed_time
    for i in p:
        print i

And the output:

5.004710
(0, 0)
(0, 1)
(0, 2)
(1, 0)
(1, 1)
(1, 2)

0.000017
(0, 0)
(0, 1)
(0, 2)
(1, 0)
(1, 1)
(1, 2)

What I would like to have as a result is, that the first delay of 5.004710 seconds is similar to the latter one (0.000017) and the blocking occurs on accessing the product elements (in the for loop).

Was it helpful?

Solution

Here is a version of product (for just two iterators), that tries to be as lazy as possible.

def product(i1, i2):
    for a in i1: 
        j2, i2 = itertools.tee(i2)
        for b in j2:
            yield (a, b)
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top