1) Add below line is yarn-site.xml
<property><name>yarn.scheduler.fair.allocation.file</name><value>/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/fair-scheduler.xml</value></property>
<property><name>yarn.acl.enable</name><value>true</value></property>
<property><name>yarn.admin.acl</name><value>mapr mapr</value></property>
<property><name>yarn.resourcemanager.scheduler.class</name><value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value></property>
2) Configure fair scheduler:
[root@mfs071 ~]# vim /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/fair-scheduler.xml
<allocations>
<queuePlacementPolicy>
<rule name="specified" create="false"/>
<rule name="primaryGroup" create="false"/>
<rule name="secondaryGroupExistingQueue" create="false"/>
<!--Create interactive queue for generic shape group?-->
<rule name="user" create="false" />
<rule name="reject" />
</queuePlacementPolicy>
<queue name="root">
<minResources>2000 mb, 1 vcores,1 disks</minResources>
<maxResources>5000 mb, 1 vcores,2 disks</maxResources>
<maxRunningApps>10</maxRunningApps>
<weight>2.0</weight>
<schedulingPolicy>fair</schedulingPolicy>
<aclSubmitApps> </aclSubmitApps>
<aclAdministerApps>root</aclAdministerApps>
<!-- <aclAdministerApps>mapr mapr</aclAdministerApps> -->
<queue name="sample_sub_queue1">
<minResources>1024 mb, 1 vcores,1 disks</minResources>
<aclSubmitApps>nitin</aclSubmitApps>
<aclAdministerApps>root</aclAdministerApps>
</queue>
<queue name="sample_sub_queue2">
<minResources>1024 mb, 1 vcores,1 disks</minResources>
<aclSubmitApps>mapr</aclSubmitApps>
<aclAdministerApps>root</aclAdministerApps>
</queue>
<queue name="sample_sub_queue3">
<minResources>1024 mb, 1 vcores,1 disks</minResources>
<aclSubmitApps>kunal</aclSubmitApps>
<aclAdministerApps>root</aclAdministerApps>
</queue>
</queue>
</allocations>
3) Restart resourcemanager
4) Login via nitin and submit job to sample_sub_queue3.
[nitin@mfs071 ~]$ yarn jar "/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0-mapr-1602.jar" teragen -Dmapred.map.tasks=20 -Dmapred.reduce.tasks=0 -Dmapred.job.queue.name=sample_sub_queue3 1000 /tmp/teragen1
Job will fail with following exception
java.io.IOException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1460957957156_0002 to YARN : User nitin cannot submit applications to queue root.sample_sub_queue3
5) Try to submit job in sample_sub_queue1. It will be successful.
6) Login via kunal and try to kill the job.
[kunal@mfs072 ~]$ yarn application -kill application_1460957957156_0001
Job will fail with following exception
Exception in thread "main" org.apache.hadoop.yarn.exceptions.YarnException: java.security.AccessControlException: User kunal cannot perform operation MODIFY_APP on application_1460957957156_0001
No comments:
Post a Comment