Multiple generators in a single loop within PHP

Question

I need to write a simple script that loads data from multiple files and merges it somehow. However, given the fact that the files might be quite huge I'd like to load data partially. To do so I decided to use yield. And according to examples I found I could use following construction for single generator:

$generator = $someClass->load(); //load method uses yield so it returns generator object
foreach($generator as $i) {
  // do something
}

But what if I want to use two generators at once?

$generatorA = $someClass1->load(); //load method uses yield so it returns generator object
$generatorB = $someClass2->load(); //load method uses yield so it returns generator object
foreach($generatorA as $i) {
  // how can I access to resultSet from generatorB here?
}

Answer 1

Generators in PHP implement the Iterator interface, so you can merge / combine multiple Generators like you can combine multiple Iterators.

If you want to iterate over both generators one after the other (merge A + B), then you can make use of the AppendIterator.

$aAndB = new AppendIterator();
$aAndB->append($generatorA);
$aAndB->append($generatorB);

foreach ($aAndB as $i) {
    ...

If you want to iterate over both generator at once, you can make use of MultipleIterator.

$both = new MultipleIterator();
$both->attachIterator($generatorA);
$both->attachIterator($generatorB);

foreach ($both as list($valueA, $valueB)) {
    ...

Example for those two incl. examples and caveats are in this blog-post of mine as well:

• Iterating over Multiple Iterators at Once

Otherwise I don't understand what you've been asking for. If you could clarify, I should be able to give you more directions.

Answer 2

From https://www.php.net/manual/en/language.generators.syntax.php#control-structures.yield.from

Generator delegation via yield from

In PHP 7, generator delegation allows you to yield values from another generator, Traversable object, or array by using the yield from keyword. The outer generator will then yield all values from the inner generator, object, or array until that is no longer valid, after which execution will continue in the outer generator.

So it's possible to combine two (or more) generators using yield from.

/**
  * Yield all values from $generator1, then all values from $generator2
  * Keys are preserved
  */
function combine_sequentially(Generator $generator1, Generator $generator2): Generator
{
    yield from $generator1;
    yield from $generator2;
};

Or something more fancy (here, it's not possible to use yield from):

/**
  * Yield a value from $generator1, then a value from $generator2, and so on
  * Keys are preserved
  */
function combine_alternatively(Generator $generator1, Generator $generator2): Generator
{
    while ($generator1->valid() || $generator2->valid()) {
        if ($generator1->valid()) {
            yield $generator1->key() => $generator1->current();
            $generator1->next();
        }
        if ($generator2->valid()) {
            yield $generator2->key() => $generator2->current();
            $generator2->next();
        }
    }
};

Answer 3

You can use yield from

  
function one()
{ 

   yield 1;
   yield 2;

}

function two()
{
   yield 3;
   yield 4;
}

function merge()
{
   yield from one();
   yield from two();

}
foreach(merge() as $i)
{
   echo $i;
}

An example Reusable function

function iterable_merge( iterable ...$iterables ): Generator {
   

    foreach ( $iterables as $iterable ) {
        
        yield from $iterable;
    }
}

$merge=iterable_merge(one(),two());

Answer 4

While AppendIterator works for Iterators, it has some issues.

Firstly it is not so nice to need to construct a new object rather than just calling a function. What is even less nice is that you need to mutate the AppendIterator, since you cannot provide the inner iterators in its constructor.

Secondly AppendIterator only takes Iterator instances, so if you have a Traversable, such as IteratorAggregate, you are out of luck. Same story for other iterable that are not Iterator, such as array.

This PHP 7.1+ function combines two iterable:

/**
 * array_merge clone for iterables using lazy evaluation
 *
 * As with array_merge, numeric elements with keys are assigned a fresh key,
 * starting with key 0. Unlike array_merge, elements with duplicate non-numeric
 * keys are kept in the Generator. Beware that when converting the Generator
 * to an array with a function such as iterator_to_array, these duplicates will
 * be dropped, resulting in identical behaviour as array_merge.
 *
 *
 * @param iterable ...$iterables
 * @return Generator
 */
function iterable_merge( iterable ...$iterables ): Generator {
    $numericIndex = 0;

    foreach ( $iterables as $iterable ) {
        foreach ( $iterable as $key => $value ) {
            yield is_int( $key ) ? $numericIndex++ : $key => $value;
        }
    }
}

Usage example:

foreach ( iterable_merge( $iterator1, $iterator2, $someArray ) as $k => $v ) {}

This function is part of a small library for working with iterable, where it is also extensively tested.

Answer 5

If you want to use Generators with AppendIterator you'll need to use NoRewindIterator with it:

https://3v4l.org/pgiXB

<?php
function foo() {
        foreach ([] as $foo) {
                yield $foo;
        }
}
$append = new AppendIterator();
$append->append(new NoRewindIterator(foo()));

var_dump(iterator_to_array($append));

Trying to traverse a bare Generator with AppendIterator will cause a fatal error if the Generator never actually calls yield:

https://3v4l.org/B4Qnh

<?php
function foo() {
        foreach ([] as $foo) {
                yield $foo;
        }
}
$append = new AppendIterator();
$append->append(foo());

var_dump(iterator_to_array($append));

Output:

Fatal error: Uncaught Exception: Cannot traverse an already closed generator in /in/B4Qnh:10
Stack trace:
#0 [internal function]: AppendIterator->rewind()
#1 /in/B4Qnh(10): iterator_to_array(Object(AppendIterator))
#2 {main}
  thrown in /in/B4Qnh on line 10

Process exited with code 255.

Answer 6

Something like:

$generatorA = $someClass1->load(); //load method uses yield so it returns generator object
$generatorB = $someClass2->load(); //load method uses yield so it returns generator object

$flag = true;
$i = 0;
while($flag === false) {

 if ($i >= count($generatorA) || $i >= count($generatorB)) {
      $flag = true;
 }

 // Access both generators
 $genA = $generatorA[$i];
 $genB = $generatorB[$i];

$i++;
}

Answer 7

Try this:

<?php
foreach($generatorA as $key=>$i) {
    $A=$i;//value from $generatorA
    $B=$generatorB[$key];//value from $generatorB
}