Question

I have a tuple containing unknown number of fields. I want to flatten the tuple so that every element of the tuple creates a new row (the way Bag flattens)

e.g.
Input:

student = { (A, B, (C, D, E, F)) }

Output:

student = { (A, B, C),
            (A, B, D),
            (A, B, E),
            (A, B, F),
          }

How do I achieve this?

Was it helpful?

Solution

OK, got it working! Used LinkedIn's DataFu UDF library function TransposeTupleToBag to transpose tuples to bags, then flattened the bag.

Something like this: out = foreach student generate $0, $1, flatten(TransposeTupleToBag($2));

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top