Tuesday, 24 November 2015

Hive configuration to access hiveserver2



<property>
<name>hive.support.concurrency</name>
<value>true</value>
</property>

<property>
<name>hive.zookeeper.quorum</name>
<value>adp031.tuxhub.com</value>
</property>

<property>
<name>hive.server2.thrift.port</name>
<value>10000</value>
</property>

Tuesday, 17 November 2015

HUE Integration with Resource manager HA



1) Edit your hue.ini add below parameters in yarn_cluster section


[[[default]]]
     security_enabled=${security_enabled}
     mechanism=${mechanism}
     history_server_api_url=http://node6.tuxhub.com:19888
     ssl_cert_ca_verify=False
     logical_name=my.cluster.com
     submit_to=True


     [[[ha]]]
     resourcemanager_host=node6.tuxhub.com
     resourcemanager_api_url=http://node6.tuxhub.com:8088
     proxy_api_url=http://node6.tuxhub.com:8088
     logical_name=my.cluster.com

     [[[ha1]]]
     resourcemanager_host=node5.tuxhub.com
     resourcemanager_api_url=http://node5.tuxhub.com:8088
     proxy_api_url=http://node5.tuxhub.com:8088
     logical_name=my.cluster.com


2) Restart hue.


Wednesday, 4 November 2015

Hive Storage Authorization




<configuration>
 <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://setup1:3306/hive?createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
</property>
 <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
 </property>
 <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    <description>username to use against metastore database</description>
 </property>
 <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>hive</value>
    <description>password to use against metastore database</description>
 </property>
 <property>
    <name>hive.metastore.uris</name>
    <value>thrift://setup1:9083</value>
 </property>
 <property><name>hive.server2.authentication</name><value>NOSASL</value></property>

<!-- SECURITY -->

<property>
    <name>hive.metastore.pre.event.listeners</name>
    <value>org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener</value>
</property>


<property>
    <name>hive.security.metastore.authorization.manager</name>
    <value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value>
</property>


<property>
    <name>hive.security.metastore.authenticator.manager</name>
    <value>org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator</value>
</property>

<property>
    <name>hive.security.authorization.manager</name>
        <value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value>
        </property>


<property>
        <name>hive.security.authorization.enabled</name>
        <value>true</value>
</property>


<property>
        <name>hive.server2.enable.doAs</name>
        <value>true</value>
</property>
<!-- SECURITY CONFIG DONE -->

Ansible Cheat sheet

Install Ansible  # yum install ansible Host file configuration  File  [ansible@kuber2 ~]$ cat /etc/ansible/hosts     [loca...