I am an absolute beginner in Hadoop, and I am only doing some simple testing, however, I do not find error messages very informative.
I have set up my Hadoop environment in a single-node mode on a CentOS 6.4 VM with 4Gb of RAM available.
I am trying to run a simple Pig script over a 500Mb CSV file. I have two 500Mb files, on the first one, the script was successful. On the second one, which is about the same size, but different data (a lot more rows), I get an error when the execution reaches about 60%.
This is the (very simple) Pig script I use:
records = LOAD 'trans2013.csv' USING PigStorage(',') AS
(podracun_v_breme,datum_transakcije,znesek_transakcije,oznaka_valute_transakcije,racun_v_dobro,naziv_prejemnika,maticna_stevilka,davcna_stevilka,sifra_pu,zr_sns_oe,namen);
transaction_recs = GROUP records ALL;
tot_trans = FOREACH transaction_recs GENERATE
SUM(records.znesek_transakcije);
STORE tot_trans INTO '/user/root/totaltransactions';
This is the error I get in the terminal:
2014-04-06 10:28:29,147 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 64% complete 2014-04-06 10:28:30,240 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 2014-04-06 10:28:30,241 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job job_1396637732046_0008 has failed! Stop running all dependent jobs 2014-04-06 10:28:30,241 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete 2014-04-06 10:28:30,460 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
recreate exception from backed error:
AttemptID:attempt_1396637732046_0008_m_000001_0 Info:Container killed
by the ApplicationMaster.
2014-04-06 10:28:30,461 [main] ERROR
org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s)
failed! 2014-04-06 10:28:30,463 [main] INFO
org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
2.0.6-alpha 0.11.1 root 2014-04-06 10:25:49 2014-04-06 10:28:30 GROUP_BY
Failed!
Failed Jobs: JobId Alias Feature Message Outputs
job_1396637732046_0008 records,tot_trans,transaction_recs GROUP_BY,COMBINER Message:
Job failed! /user/root/totaltransactions,
Input(s): Failed to read data from
"hdfs://localhost:8020/user/root/trans2013.csv"
Output(s): Failed to produce result in "/user/root/totaltransactions"
Counters: Total records written : 0 Total bytes written : 0 Spillable
Memory Manager spill count : 0 Total bags proactively spilled: 0 Total
records proactively spilled: 0
Job DAG: job_1396637732046_0008
2014-04-06 10:28:30,463 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed! 2014-04-06 10:28:30,491 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2997: Unable to
recreate exception from backed error:
AttemptID:attempt_1396637732046_0008_m_000001_0 Info:Container killed
by the ApplicationMaster.
Details at logfile: /root/pig_1396797945352.log
Here is the error from the log:
Backend error message
--------------------- AttemptID:attempt_1396637732046_0008_m_000001_0 Info:Container killed by the ApplicationMaster.
Pig Stack Trace
--------------- ERROR 2997: Unable to recreate exception from backed error: AttemptID:attempt_1396637732046_0008_m_000001_0 Info:Container
killed by the ApplicationMaster.
org.apache.pig.backend.executionengine.ExecException: ERROR 2997:
Unable to recreate exception from backed error:
AttemptID:attempt_1396637732046_0008_m_000001_0 Info:Container killed
by the ApplicationMaster.
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:217)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:149)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:400)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1264) at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
at org.apache.pig.PigServer.execute(PigServer.java:1239) at
org.apache.pig.PigServer.executeBatch(PigServer.java:333) at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:137)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84) at
org.apache.pig.Main.run(Main.java:604) at
org.apache.pig.Main.main(Main.java:157) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.apache.hadoop.util.RunJar.main(RunJar.java:212)