Mahout : Cannot convert into sequence file

StackOverflow https://stackoverflow.com/questions/22130500

Question

I'm trying to convert some text files into mahout sequence files. So I do 

mahout seqdirectory -i inputFolder -o outputFolder

But I always get this exception

java.lang.Exception: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)
Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
    at org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader.initNextRecordReader(CombineFileRecordReader.java:164)
    at org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader.<init>(CombineFileRecordReader.java:126)
    at org.apache.mahout.text.MultipleTextFileInputFormat.createRecordReader(MultipleTextFileInputFormat.java:43)
    at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:491)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:734)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:235)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader.initNextRecordReader(CombineFileRecordReader.java:155)
    ... 11 more
Caused by: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
    at org.apache.mahout.text.WholeFileRecordReader.<init>(WholeFileRecordReader.java:52)
    ... 16 more

I'm running Mahout 0.8 on Hadoop 2.2.0

Any ideas ?

Was it helpful?

Solution

The previous answers are incorrect. Mahout 0.8 had a MapReduce version of seqdirectory which was a new feature. A bug in the MR version was causing the exception you are seeing.

To execute seqdirectory with Mahout 0.8, please use the sequential version by specifying the -xm sequential option to your command line.

mahout seqdirectory -i inputFolder -o outputFolder -xm sequential

By default seqdirectory executes the MR version if none specified.

This issue has since been fixed in Mahout 0.9.

OTHER TIPS

As I read somewhere that mahout 0.8 works with hadoop 1.2. I only downloaded mahout (it uses hadoop jar from lib/hadoop )

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top