Question

can i use iterable traverse the list again and again? when i use ArrayList,i can traverse the list again and again in "foreach" ,but when i use iterable as a parameter in a function in hadoop(a distributed compute framework),only the first time i can traverse the iterable, when i use foreach again , it get nonthing. ex:

public void reduce(Text key, Iterable<Text> values, Context context) 
throws IOException, InterruptedException {

float all=0;
String resultKey;
float resultValue;
ArrayList<String> valuelist=new ArrayList<String>();

for (Text text : values) {
    valuelist.add(text.toString());
}

for (String text : valuelist) {
    String[] contents=text.toString().split(" ");
    if(contents.length==1)
    {
        all=Float.parseFloat(contents[0]);
        break;
    }
}

if(all==0)
{
    return;
}

for (String text : valuelist) {
    String[] contents=text.toString().split(" ");

    if(contents.length>1)
    {
        resultKey=contents[0]+" "+key.toString();
        resultValue=Float.parseFloat(contents[1])/all;

        context.write(new Text(resultKey), new Text(resultValue+""));
    }
}
}

-----i have to save it in the ArrayList first... in my understanding, foreach only need an iterable, why ArrayList can,but the parameter can't? thanks for reading so much.

Was it helpful?

Solution

This isn't a problem with Iterable in general, but perhaps it's a problem with the specific implementation class that's being passed to that method by the framework. If you think about it, that class is probably pulling each element of the Iterable over the network right when you call next(); that would explain why you can't just run it over again. Saving each element is a good solution if that's really what you need to do.

OTHER TIPS

I think the issue has to do with iterators being one-way, one-time kinds of collections. From the google-collections faq:

An Iterator represents a one-way scrollable "stream" of elements, and an Iterable is anything which can spawn independent iterators. A Collection is much, much more than this, so we only require it when we need to.1

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top