How can I cannot index into the values list of reduce?
Question
I am using in-mapper combining in a Map Reduce job via the Python mrjob module. Because I wrote a mapper_final function that emits a single pair, I am sure that only a single key-value pair is emitted to my reducers.
However, my reduce function is erring:
def reducer(self, key, occurrences):
'''
Calculates the final value.
'''
yield 'Final Value: ', occurrences[0] / 2
The error reads
File "calculateFinalValue.py", line 354, in reducer
yield 'Final Value: ', occurrences[0] / 2
TypeError: 'generator' object has no attribute '__getitem__'
Why can I not index into occurrences
? There should only be a single pair in that list, right?
Solution
occurrences
is not a list
, it is a generator
. If you want a list
, you need to assemble the generator results into a list. Something like:
list_occurrences = [ occ for occ in occurrences ]
or
list_occurrences = list(occurrences)
yield 'Final Value: ', list_occurrences[0] / 2
Or you can get the first value of occurrences with occurrences.next()
:
yield 'Final Value: ', occurrences.next() / 2
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow