Я установил oozie 4.3.0 в hadoop 2.7.3. Службы oozie и hadoop запущены и запущены. Я попытался выполнить примерную работу по уменьшению количества карт, которая поставляется с дистрибутивом oozie, когда я проверяю журнал задания, он содержит эти ошибки.Ошибка в примере на карте уменьшить работу с помощью java.lang.NoSuchFieldError: HADOOP_CLASSPATH
2016-12-20 10:46:11,615 INFO ActionStartXCommand:520 - SERVER[data01.teg.io] USER[hadoop] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-161220104605103-oozie-hado-W] ACTION[[email protected]:start:] Start action [[email protected]:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-12-20 10:46:11,615 INFO ActionStartXCommand:520 - SERVER[data01.teg.io] USER[hadoop] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-161220104605103-oozie-hado-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action status=DONE
2016-12-20 10:46:11,616 INFO ActionStartXCommand:520 - SERVER[data01.teg.io] USER[hadoop] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-161220104605103-oozie-hado-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action updated in DB!
2016-12-20 10:46:11,912 INFO WorkflowNotificationXCommand:520 - SERVER[data01.teg.io] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-161220104605103-oozie-hado-W] ACTION[[email protected]:start:] No Notification URL is defined. Therefore nothing to notify for job [email protected]:start:
2016-12-20 10:46:11,921 INFO WorkflowNotificationXCommand:520 - SERVER[data01.teg.io] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-161220104605103-oozie-hado-W] ACTION[] No Notification URL is defined.Therefore nothing to notify for job 0000000-161220104605103-oozie-hado-W
2016-12-20 10:46:11,959 INFO ActionStartXCommand:520 - SERVER[data01.teg.io] USER[hadoop] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-161220104605103-oozie-hado-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-12-20 10:46:12,146 INFO HadoopAccessorService:520 - SERVER[data01.teg.io] USER[hadoop] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-161220104605103-oozie-hado-W] ACTION[[email protected]] Processing configuration file [/srv/oozie-4.3.0/distro/target/oozie-4.3.0-distro/oozie-4.3.0/conf/action-conf/default.xml] for action [default] and hostPort [*]
2016-12-20 10:46:12,148 INFO HadoopAccessorService:520 - SERVER[data01.teg.io] USER[hadoop] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-161220104605103-oozie-hado-W] ACTION[[email protected]] Processing configuration file [/srv/oozie-4.3.0/distro/target/oozie-4.3.0-distro/oozie-4.3.0/conf/action-conf/map-reduce.xml] for action [map-reduce] and hostPort [*]
2016-12-20 10:46:13,232 ERROR ActionStartXCommand:517 - SERVER[data01.teg.io] USER[hadoop] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-161220104605103-oozie-hado-W] ACTION[[email protected]] Error,
java.lang.NoSuchFieldError: HADOOP_CLASSPATH
at org.apache.hadoop.mapreduce.v2.util.MRApps.setClasspath(MRApps.java:248)
at org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:458)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:285)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:240)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1183)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1369)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
at org.apache.oozie.command.XCommand.call(XCommand.java:287)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Я искал каждую ссылку в google, но ни один из предлагаемых обходных решений/решений не работал. Любой может понять, почему эта ошибка происходит.
Я могу запускать работу по уменьшению количества заданий без каких-либо проблем, и я могу видеть детали пути к классам с помощью команды «hasoop classpath» в командной строке оболочки.
Мои job.properties файл
nameNode=hdfs://localhost:54310
jobTracker=localhost:8032
queueName=default
examplesRoot=examples
oozie.use.system.libpath=true
oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}apps/map-reduce/workflow.xml
outputDir=map-reduce
И workflow.xml не имеет каких-либо изменений только стандартный файл поставляется с распределением oozie.
файл конфигурации oozie-site.xml является
<configuration>
<!--
Refer to the oozie-default.xml file for the complete list of
Oozie configuration properties and their default values.
-->
<!-- Proxyuser Configuration -->
<property>
<name>oozie.service.ProxyUserService.proxyuser.hadoop.hosts</name>
<value>*</value>
<description>
List of hosts the '#USER#' user is allowed to perform 'doAs'
operations.
The '#USER#' must be replaced with the username o the user who is
allowed to perform 'doAs' operations.
The value can be the '*' wildcard or a list of hostnames.
For multiple users copy this property and replace the user name
in the property name.
</description>
</property>
<property>
<name>oozie.service.ProxyUserService.proxyuser.hadoop.groups</name>
<value>*</value>
<description>
List of groups the '#USER#' user is allowed to impersonate users
from to perform 'doAs' operations.
The '#USER#' must be replaced with the username o the user who is
allowed to perform 'doAs' operations.
The value can be the '*' wildcard or a list of groups.
For multiple users copy this property and replace the user name
in the property name.
</description>
</property>
<property>
<name>oozie.service.WorkflowAppService.system.libpath</name>
<value>hdfs:///user/hadoop/share/lib/</value>
<description>
System library path to use for workflow applications.
This path is added to workflow application if their job properties sets
the property 'oozie.use.system.libpath' to true.
</description>
</property>
<property>
<name>oozie.action.retry.interval</name>
<value>1</value>
<description>
The interval between retries of an action in case of failure
</description>
</property>
<property>
<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
<value>*=/srv/hadoop-2.7.3/etc/hadoop</value>
<description>
Comma separated AUTHORITY=HADOOP_CONF_DIR, where AUTHORITY is the HOST:PORT of
the Hadoop service (JobTracker, YARN, HDFS). The wildcard '*' configuration is
used when there is no exact match for an authority. The HADOOP_CONF_DIR contains
the relevant Hadoop *-site.xml files. If the path is relative is looked within
the Oozie configuration directory; though the path can be absolute (i.e. to point
to Hadoop client conf/ directories in the local filesystem.
</description>
</property>
</configuration>
И команда используется для запуска
$ bin/oozie job -oozie http://localhost:11000/oozie -config /srv/oozie-4.3.0/distro/target/oozie-4.3.0-distro/oozie-4.3.0/examples/apps/map-reduce/job.properties -run
$ bin/oozie job -oozie http://localhost:11000/oozie -log 0000003-161218192921381-oozie-hado-W
@youngHobbit: Я не могу интерпретировать ваш отрицательный голос, сообщить мне, какие детали вы ожидаете или что помогает понять? – Marsi
Я не спускался с вашего сообщения. Я посмотрю ваше сообщение позже. Благодарю. – YoungHobbit