質問

I have a HaspMap which has "Text" objects as the key and "Integer" Object as Values. The Value is actually the number of Occurrences of the Key in my code. So for the first time it will be 1 and then it keeps incrementing. The Code is shown below.

First I check if a given "Text" Object exists in the Map. If it does not then I add it as a Key to the Map along with a Value of 1. But the problem that I am facing is that when I add the new Key and value to the map through "put" function for some reason all the previously present Key/value pairs in the Map are getting replaced with the new One. The code is given below.

public class WordPatternReducer extends Reducer<IntWritable,Text, Text, IntWritable>{
private IntWritable totalWordCount = new IntWritable();
private Map<Text,Integer> valueCount=new HashMap<Text,Integer>();
private Map<IntWritable,HashMap<Text,Integer>> posMap=new HashMap<IntWritable, HashMap<Text, Integer>>(); 
 @Override
 public void reduce(IntWritable key, Iterable<Text> values, Context context)
            throws IOException, InterruptedException {
  Iterator<Text> it=values.iterator();
  Integer maxCountInteger=new Integer(1);
  Text maxOccurText = null;
  Text newval=new Text();
  while (it.hasNext()) {
      newval=it.next();
      System.out.println("The new val outside is"+newval);
      if (valueCount.containsKey(newval)) {
          System.out.println("The new val inside if is"+newval);
          valueCount.put(newval, (valueCount.get(newval)+1));
      } else {
          System.out.println(newval);
          valueCount.put(newval, 1);
          System.out.println(valueCount.toString());
      }
      maxOccurText=newval;

 } }}

So the check for existing keys is working as it always go to the else statement. But the values are getting replaced. In the output console I am getting the following output.

The new val outside isWelcome
Welcome
{Welcome=1}
The new val outside isservice service
{service=1, service=1}
The new val outside isversions     versions
{versions=1, versions=1, versions=1}
The new val outside isto
to
{to=1, to=1, to=1, to=1}
The new val outside isproviders, 
providers,
{providers,=1, providers,=1, providers,=1, providers,=1, providers,=1}
The new val outside isof of
{of=1, of=1, of=1, of=1, of=1, of=1}
The new val outside isthe   the
{the=1, the=1, the=1, the=1, the=1, the=1, the=1}
The new val outside issome    some
{some=1, some=1, some=1, some=1, some=1, some=1, some=1, some=1}

and so on..

I do not want this. I just want to add the new Key value pairs retaining old one. Could someone please let me know what I have done wrong? Thank you in advance.

役に立ちましたか?

解決

You're storing a reference to newval in the map. The containsKey(newval) checks false because Text checks to see if its contents are the same, but then the put call stores a reference to same newval that you've put in the map repeatedly (which contains only the latest string read in). Try changing the map to a map of String,Int and calling map.put(newval.toString()) to start, which should lead you to a better solution. Else, declare newval within the iterator loop so that you're storing a new Text object in the map each time (Text newval = it.next()).

他のヒント

What class is Text?

Hypothesis 1: Text doesn't implement equals() and hashCode() according to the contract. That'll mess up a map.

Hypothesis 2: Text is mutable and changing during the iteration. Using String keys in the map would fix that.

It looks like your Text value is mutable. What is the implementation of the Iterable values object? Is it a parser that repeatedly returns the same token instance, modifying its type and text with each call to next()? If so, you'll need to copy the text to a new immutable object—which might simply be a String—and use that immutable token as your key.

ライセンス: CC-BY-SA帰属
所属していません StackOverflow
scroll top