Question

My hama throws the following exception during the input data partition phase before actually running my BSP job. Can I know what are the possible root causes of this exception? Any suggestions about how to find out the root cause is appreciated. Thank you!

13/11/06 03:50:50 WARN snappy.LoadSnappy: Snappy native library not loaded
13/11/06 03:50:50 INFO sync.ZKSyncClient: Initializing ZK Sync Client
13/11/06 03:50:50 INFO sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At masked-addr:33960
13/11/06 03:50:52 ERROR bsp.BSPTask: Error running bsp setup and bsp function.
java.lang.NullPointerException
    at java.lang.Class.isAssignableFrom(Native Method)
    at org.apache.hadoop.io.serializer.WritableSerialization.accept(WritableSerialization.java:100)
    at org.apache.hadoop.io.serializer.SerializationFactory.getSerialization(SerializationFactory.java:83)
    at org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73)
    at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:963)
    at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:896)
    at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:393)
    at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:284)
    at org.apache.hama.bsp.PartitioningRunner.bsp(PartitioningRunner.java:217)
    at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:177)
    at org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
    at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246)
13/11/06 03:50:52 INFO ipc.Server: Stopping server on 33960
13/11/06 03:50:52 INFO ipc.Server: IPC Server handler 0 on 33960: exiting
13/11/06 03:50:52 INFO ipc.Server: Stopping IPC Server listener on 33960
Était-ce utile?

La solution

Found the root cause. This exception happens when at least one of the input files specified in the input paths is size 0.

Licencié sous: CC-BY-SA avec attribution
Non affilié à StackOverflow
scroll top