l'application hadoop échoue avec le succès de mapreduce
https://stackoverflow.com//questions/24043178
-
21-12-2019 - |
italiano
english
français
española
中国
日本の
العربية
Deutsch
한국어
Português
RussianQuestion
Je suis relativement nouveau sur hadoop 2 (hadoop 2.2.0) et je ne comprends pas pourquoi l'application M/R job ~ sur le gestionnaire de ressources est marquée comme ayant échoué :
application_1399458460502_0015 pig Max temperature MAPREDUCE default Wed, 04 Jun 2014 17:16:52 GMT Wed, 04 Jun 2014 17:17:30 GMT FAILED FAILED History
Quand je sais que le travail M/R s'est terminé avec succès et que même le serveur d'historique des travaux prétend qu'il a réussi :
2014.06.04 13:16:52 EDT 2014.06.04 13:17:19 EDT job_1399458460502_0015 Max temperature pig default SUCCEEDED 2 2 1 1
Je ne comprends pas pourquoi les candidatures ont été marquées comme ayant échoué.La seule erreur que je vois sur les journaux du serveur JobHistory est la suivante :
2014-06-04 13:17:19,628 INFO [Thread-62] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true
2014-06-04 13:17:19,628 INFO [Thread-62] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true
2014-06-04 13:17:19,628 INFO [Thread-62] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services
2014-06-04 13:17:19,629 INFO [Thread-62] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0
2014-06-04 13:17:19,736 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://namenodeha/user/pig/.staging/job_1399458460502_0015/job_1399458460502_0015_1.jhist to hdfs://namenodeha/mr-history/tmp/pig/job_1399458460502_0015-1401902212831-pig-Max+temperature-1401902239623-2-1-SUCCEEDED-default.jhist_tmp
2014-06-04 13:17:19,812 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://namenodeha/mr-history/tmp/pig/job_1399458460502_0015-1401902212831-pig-Max+temperature-1401902239623-2-1-SUCCEEDED-default.jhist_tmp
2014-06-04 13:17:19,824 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://namenodeha/user/pig/.staging/job_1399458460502_0015/job_1399458460502_0015_1_conf.xml to hdfs://namenodeha/mr-history/tmp/pig/job_1399458460502_0015_conf.xml_tmp
2014-06-04 13:17:19,835 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:1 CompletedMaps:2 CompletedReds:1 ContAlloc:3 ContRel:0 HostLocal:2 RackLocal:0
2014-06-04 13:17:19,880 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://namenodeha/mr-history/tmp/pig/job_1399458460502_0015_conf.xml_tmp
2014-06-04 13:17:19,914 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://namenodeha/mr-history/tmp/pig/job_1399458460502_0015.summary_tmp to hdfs://namenodeha/mr-history/tmp/pig/job_1399458460502_0015.summary
2014-06-04 13:17:19,925 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://namenodeha/mr-history/tmp/pig/job_1399458460502_0015_conf.xml_tmp to hdfs://namenodeha/mr-history/tmp/pig/job_1399458460502_0015_conf.xml
2014-06-04 13:17:19,937 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://namenodeha/mr-history/tmp/pig/job_1399458460502_0015-1401902212831-pig-Max+temperature-1401902239623-2-1-SUCCEEDED-default.jhist_tmp to hdfs://namenodeha/mr-history/tmp/pig/job_1399458460502_0015-1401902212831-pig-Max+temperature-1401902239623-2-1-SUCCEEDED-default.jhist
2014-06-04 13:17:19,938 INFO [Thread-62] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
2014-06-04 13:17:19,940 INFO [Thread-62] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job diagnostics to
2014-06-04 13:17:20,060 ERROR [Thread-62] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Exception while unregistering
java.lang.NullPointerException
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.getApplicationWebURLOnJHSWithoutScheme(MRWebAppUtil.java:133)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.getApplicationWebURLOnJHSWithScheme(MRWebAppUtil.java:148)
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.doUnregistration(RMCommunicator.java:207)
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.unregister(RMCommunicator.java:177)
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStop(RMCommunicator.java:250)
at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStop(RMContainerAllocator.java:255)
at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStop(MRAppMaster.java:817)
at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at org.apache.hadoop.service.CompositeService.stop(CompositeService.java:159)
at org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:132)
at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:548)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:599)
2014-06-04 13:17:20,061 INFO [Thread-62] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:1 CompletedMaps:2 CompletedReds:1 ContAlloc:3 ContRel:0 HostLocal:2 RackLocal:0
2014-06-04 13:17:20,062 INFO [Thread-62] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Skipping cleaning up the staging dir. assuming AM will be retried.
2014-06-04 13:17:20,062 INFO [Thread-62] org.apache.hadoop.ipc.Server: Stopping server on 43851
2014-06-04 13:17:20,064 INFO [IPC Server listener on 43851] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 43851
2014-06-04 13:17:20,065 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2014-06-04 13:17:20,065 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted
2014-06-04 13:17:25,066 INFO [Thread-62] org.apache.hadoop.ipc.Server: Stopping server on 44771
2014-06-04 13:17:25,066 INFO [IPC Server listener on 44771] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 44771
2014-06-04 13:17:25,067 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2014-06-04 13:17:25,072 INFO [Thread-62] org.mortbay.log: Stopped SelectChannelConnector@0.0.0.0:0
2014-06-04 13:17:25,172 INFO [Thread-62] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Exiting MR AppMaster..GoodBye!
2014-06-04 13:17:25,173 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster received a signal. Signaling RMCommunicator and JobHistoryEventHandler.
2014-06-04 13:17:25,173 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator notified that iSignalled is: true
2014-06-04 13:17:25,173 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: false
2014-06-04 13:17:25,173 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator notified that shouldUnregistered is: false
2014-06-04 13:17:25,173 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: false
2014-06-04 13:17:25,174 INFO [Thread-1] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is false
Eroor du gestionnaire de ressources :
Application application_1399458460502_0015 failed 2 times due to AM Container for appattempt_1399458460502_0015_000002 exited with exitCode: 1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
.Failing this attempt.. Failing the application
Aucune de ces erreurs ne me donne une idée de ce qui se passe ici.Ma configuration est la suivante :
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<!--internal property-->
<property>
<name>bigdata.conf.identification</name>
<value>cluster-DEV1</value>
<final>true</final>
</property>
<!--hadoop properties-->
<!-- Put site-specific property overrides in this file. -->
<!--hbase-site-->
<property>
<name>hbase.zookeeper.quorum</name>
<value>bd-prg-dev1-nn1,bd-prg-dev1-nn2,bd-prg-dev1-rm1</value>
</property>
<property>
<name>zookeeper.session.timeout</name>
<value>60000</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>
<property>
<name>zookeeper.znode.parent</name>
<value>/hbase-unsecure</value>
</property>
<!--core-site-->
<property>
<name>hadoop.security.authentication</name>
<value>simple</value>
</property>
<property>
<name>ipc.client.connect.max.retries</name>
<value>50</value>
</property>
<property>
<name>ipc.client.connection.maxidletime</name>
<value>30000</value>
</property>
<property>
<name>ipc.client.idlethreshold</name>
<value>8000</value>
</property>
<property>
<name>io.compression.codecs</name>
<value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec</value>
</property>
<property>
<name>io.serializations</name>
<value>org.apache.hadoop.io.serializer.WritableSerialization</value>
</property>
<property>
<name>hadoop.security.authorization</name>
<value>false</value>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://namenodeha</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>bd-prg-dev1-nn1:2181,bd-prg-dev1-nn2:2181,bd-prg-dev1-rm1:2181</value>
</property>
<!-- hdfs-site-->
<property>
<name>dfs.namenode.http-address</name>
<value>bd-prg-dev1-nn1:50070</value>
</property>
<property>
<name>dfs.datanode.ipc.address</name>
<value>0.0.0.0:8010</value>
</property>
<property>
<name>dfs.journalnode.http-address</name>
<value>0.0.0.0:8480</value>
</property>
<property>
<name>dfs.namenode.accesstime.precision</name>
<value>0</value>
</property>
<property>
<name>dfs.namenode.stale.datanode.interval</name>
<value>30000</value>
</property>
<property>
<name>dfs.datanode.address</name>
<value>0.0.0.0:50010</value>
</property>
<property>
<name>dfs.datanode.http.address</name>
<value>0.0.0.0:50075</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>bd-prg-dev1-nn2:50090</value>
</property>
<property>
<name>dfs.nameservices</name>
<value>namenodeha</value>
<description>Logical name for this new nameservice</description>
</property>
<property>
<name>dfs.ha.namenodes.namenodeha</name>
<value>nn1,nn2</value>
<description>Unique identifiers for each NameNode in the nameservice</description>
</property>
<property>
<name>dfs.namenode.rpc-address.namenodeha.nn1</name>
<value>bd-prg-dev1-nn1:8020</value>
</property>
<property>
<name>dfs.namenode.rpc-address.namenodeha.nn2</name>
<value>bd-prg-dev1-nn2:8020</value>
</property>
<property>
<name>dfs.namenode.http-address.namenodeha.nn1</name>
<value>bd-prg-dev1-nn1:50070</value>
</property>
<property>
<name>dfs.namenode.http-address.namenodeha.nn2</name>
<value>bd-prg-dev1-nn2:50070</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.namenodeha</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<!--yarn-site-->
<property>
<name>yarn.nodemanager.address</name>
<value>0.0.0.0:45454</value>
</property>
<property>
<name>yarn.nodemanager.container-monitor.interval-ms</name>
<value>3000</value>
</property>
<property>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/app-logs</value>
</property>
<property>
<name>yarn.log.server.url</name>
<value>bd-prg-dev1-rm1:19888/jobhistory/logs</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>bd-prg-dev1-rm1:8141</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>bd-prg-dev1-rm1:8025</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.log-aggregation.compression-type</name>
<value>gz</value>
</property>
<property>
<name>yarn.nodemanager.health-checker.script.path</name>
<value>/etc/hadoop/conf/health_check</value>
</property>
<property>
<name>yarn.nodemanager.container-executor.class</name>
<value>org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor</value>
</property>
<property>
<name>yarn.nodemanager.remote-app-log-dir-suffix</name>
<value>logs</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>bd-prg-dev1-rm1:8088</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>bd-prg-dev1-rm1:8050</value>
</property>
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>bd-prg-dev1-rm1:8030</value>
</property>
<!--mapred-site-->
<property>
<name>mapreduce.jobhistory.intermediate-done-dir</name>
<value>/mr-history/tmp</value>
</property>
<property>
<name>mapreduce.jobhistory.done-dir</name>
<value>/mr-history/done</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>bd-prg-dev1-rm1:19888</value>
</property>
<property>
<name>mapreduce.jobtracker.system.dir</name>
<value>/mapred/system</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
Sortie de la console client :
[pig@hadooptools Hadoop]$ hadoop jar Jobs/hadoop-mr-poc-1.0-SNAPSHOT.jar com.ncr.bigdata.mr.MaxTemperatureDriver -conf clientConf/cluster/hadoop-client.xmlMRtest/input /MRtest/output/1
14/06/04 12:17:37 INFO mr.MaxTemperatureDriver: Job Max temperature submitted
14/06/04 12:17:37 INFO client.RMProxy: Connecting to ResourceManager at bd-prg-dev1-rm1/153.86.208.210:8050
14/06/04 12:17:37 INFO client.RMProxy: Connecting to ResourceManager at bd-prg-dev1-rm1/153.86.208.210:8050
14/06/04 12:17:38 INFO mapred.FileInputFormat: Total input paths to process : 1
14/06/04 12:17:38 INFO net.NetworkTopology: Adding a new node: /default-rack/153.86.208.208:50010
14/06/04 12:17:38 INFO net.NetworkTopology: Adding a new node: /default-rack/153.86.208.249:50010
14/06/04 12:17:39 INFO mapreduce.JobSubmitter: number of splits:2
14/06/04 12:17:39 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
14/06/04 12:17:39 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
14/06/04 12:17:39 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
14/06/04 12:17:39 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
14/06/04 12:17:39 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
14/06/04 12:17:39 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/06/04 12:17:39 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
14/06/04 12:17:39 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
14/06/04 12:17:39 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
14/06/04 12:17:39 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1399458460502_0015
14/06/04 12:17:39 INFO impl.YarnClientImpl: Submitted application application_1399458460502_0015 to ResourceManager at bd-prg-dev1-rm1/153.86.208.210:8050
14/06/04 12:17:39 INFO mapreduce.Job: The url to track the job: http://bd-prg-dev1-rm1:8088/proxy/application_1399458460502_0015/
14/06/04 12:17:39 INFO mapreduce.Job: Running job: job_1399458460502_0015
14/06/04 12:17:46 INFO mapreduce.Job: Job job_1399458460502_0015 running in uber mode : false
14/06/04 12:17:46 INFO mapreduce.Job: map 0% reduce 0%
14/06/04 12:17:59 INFO mapreduce.Job: map 100% reduce 0%
14/06/04 12:18:07 INFO mapreduce.Job: map 100% reduce 100%
14/06/04 12:18:31 INFO ipc.Client: Retrying connect to server: bd-prg-dev1-dn1/153.86.208.208:44771. Already tried 0 time(s); maxRetries=45
14/06/04 12:18:51 INFO ipc.Client: Retrying connect to server: bd-prg-dev1-dn1/153.86.208.208:44771. Already tried 1 time(s); maxRetries=45
14/06/04 12:19:11 INFO ipc.Client: Retrying connect to server: bd-prg-dev1-dn1/153.86.208.208:44771. Already tried 2 time(s); maxRetries=45
14/06/04 12:33:14 INFO ipc.Client: Retrying connect to server: bd-prg-dev1-dn1/153.86.208.208:44771. Already tried 44 time(s); maxRetries=45
14/06/04 12:33:35 INFO mapreduce.Job: map 0% reduce 0%
14/06/04 12:33:35 INFO mapreduce.Job: Job job_1399458460502_0015 failed with state FAILED due to: Application application_1399458460502_0015 failed 2 times due to AM Container for appattempt_1399458460502_0015_000002 exited with exitCode: 1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
.Failing this attempt.. Failing the application.
14/06/04 12:33:35 INFO mapreduce.Job: Counters: 0
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)
at com.ncr.bigdata.mr.MaxTemperatureDriver.run(MaxTemperatureDriver.java:46)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at com.ncr.bigdata.mr.MaxTemperatureDriver.main(MaxTemperatureDriver.java:54)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Quelqu'un a-t-il deviné quel pourrait être le problème ?
La solution
Je ne suis pas exactement sûr de la propriété de configuration, mais quand j'ai pris la cofiguration du cluster et créer un objet de configuration de celui qu'il ne fonctionne tout simplement bien.
Autres conseils
Ce type de problème semble se produire lorsqu'un élément de la configuration est introuvable.(Comme cela est laissé entendre dans la réponse originale).
Dans mon cas, j'ai découvert que lorsque je faisais une action de cochon, je devais me référer à un site.xml, la partie pertinente de mon flux de travail ressemble maintenant à ceci :
<action name="read-into-table">
<pig>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>site.xml</job-xml>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<script>myFile.pig</script>
</pig>
<ok to="end" />
<error to="fail" />
</action>
Et il manquait la ligne suivante :
<job-xml>site.xml</job-xml>
Licencié sous: CC-BY-SA avec attribution
Non affilié à StackOverflow
