Hashmap in Hadoop

Question

I would like to use the hashmap class defined in java.util.* in hadoop program.

A hashmap in Java is instantiated using Map D = new HashMap();

When you use the above line and compile with hadoop jar , it throws an error as Map is used for something different in hadoop.

On searching for hadoop equivalent of hashmap , I stumbled on HashMapWritable

http://lintool.github.io/Cloud9/docs/api/edu/umd/cloud9/io/map/HashMapWritable.html

One example program where HashMapWritable is used looks like this :

http://trac.nchc.org.tw/cloud/browser/sample/hadoop-0.16/tw/org/nchc/tuple/HashMapWritable.java

I don't want to go through the pain of creating a separate class and defining separate objects for keys and values.

Is there a simpler way , I can implement this in a single line like we do in Java ( I mean in a simple way like hashmap(key , value) H = new hashmap() )

Answer 1

You can try:

java.util.Map<K,V> map= new HashMap<K,V>();

that way it will resolve to the Map class that you want.

Answer 2

Remember one thing : You cannot directly write custom class objects on hadoop file system. You need to use their equivalent writable classes to store them onto HDFS. For example : You need to use Text class instead of String(Text is writable version of String). Similarly, if your map contains custom objects you need to create a writable class and handle the logic of writing each and every element of it.