Wednesday, 23 December 2015

Configure HUE with OpenLdap



Step 1:

Add below lines in ldap section in hue.ini

[[ldap]]
    base_dn="dc=tuxhub,dc=com"
    ldap_url=ldap://adp034.tuxhub.com:389
    use_start_tls=false
    bind_dn="uid=nitin,ou=Users,dc=tuxhub,dc=com"
    bind_password=nitin
    ldap_username_pattern="uid=<username>,ou=Users,dc=tuxhub,dc=com"
    search_bind_authentication=false





Thursday, 17 December 2015

Impala openldap configuration



Step 1:

Add below line in impala env.sh


[root@mfs022 ~]# vim /opt/mapr/impala/impala-1.4.1/conf/env.sh
 
   IMPALA_SERVER_ARGS=
 
    -enable_ldap_auth \
    -ldap_uri ldap://adp034.tuxhub.com:389 \
    -ldap_bind_pattern "uid=#UID,ou=Users,dc=tuxhub,dc=com" \


Step 2 :

Connect to imapla shell 

[root@mfs022 ~]# impala-shell -l -u nitin
Starting Impala Shell using LDAP-based authentication
LDAP password for nitin:
Connected to mfs022.tuxhub.com:21000
Server version: impalad version 1.4.1 RELEASE (build 2b626c8e9f4c666d23872c228cf43daae4c9acbb)
Welcome to the Impala shell. Press TAB twice to see a list of available commands.

Copyright (c) 2012 Cloudera, Inc. All rights reserved.

(Shell build version: Impala Shell v1.4.1 (2b626c8) built on Thu Feb  5 14:53:44 PST 2015)
[mfs022.tuxhub.com:21000] >

Saturday, 5 December 2015

Impala through a Proxy for High Availability



Step 1
       
       Installing HAProxy

       [root@mfs021 ~]# yum install haproxy

Steps 2

      Edit Config file for haproxy

      [root@mfs021 ~]# cat /etc/haproxy/haproxy.cfg

#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
##---------------------------------------------------------------------

defaults
    mode                    tcp
    log                     global
    retries                 3
    maxconn                 3000
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend  main *:5000
    acl url_static       path_beg       -i /static /images /javascript /stylesheets
    acl url_static       path_end       -i .jpg .gif .png .css .js

    use_backend static          if url_static
    default_backend             impala

#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
backend static
    balance     roundrobin
    server      static 127.0.0.1:4331 check

#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------

backend impala
    mode tcp
    option tcplog
    balance leastconn

    server impala1 mfs022.tuxhub.com:21000
    server impala2 mfs023.tuxhub.com:21000

Step 3 :

       Connect to impala:

root@mfs021 ~]# impala-shell
Starting Impala Shell without Kerberos authentication
Error connecting: TTransportException, Could not connect to mfs021.tuxhub.com:21000
Welcome to the Impala shell. Press TAB twice to see a list of available commands.

Copyright (c) 2012 Cloudera, Inc. All rights reserved.

(Shell build version: Impala Shell v1.4.1 (2b626c8) built on Thu Feb  5 14:53:44 PST 2015)
[Not connected] > connect localhost:5000;
Connected to localhost:5000
Server version: impalad version 1.4.1 RELEASE (build 2b626c8e9f4c666d23872c228cf43daae4c9acbb)
[localhost:5000] > show tables;

     
  

Wednesday, 2 December 2015

Apache sentry configuration on MapR Hadoop



Step  1) Please add below property in hive-site.xml

[root@mfs021 ~]# vim /opt/mapr/hive/hive-0.13/conf/hive-site.xml

<property>
  <name>hive.server2.session.hook</name>
  <value>org.apache.sentry.binding.hive.HiveAuthzBindingSessionHook</value>
</property>

<property>
  <name>hive.sentry.conf.url</name>
   <value>file:///opt/mapr/sentry/sentry-1.4.0/conf/sentry-site.xml</value>
 </property>

<property>
  <name>hive.security.authorization.task.factory</name>
  <value>org.apache.sentry.binding.hive.SentryHiveAuthorizationTaskFactoryImpl</value>
</property>

<property>
  <name> hive.metastore.execute.setugi </name>
   <value> true </value>
 </property>

Step 2) Please add below property in sentry-site.xml

[root@mfs021 ~]# vim /opt/mapr/sentry/sentry-1.4.0/conf/sentry-site.xml


<property>
        <name>sentry.hive.provider.backend</name>
        <value>org.apache.sentry.provider.file.SimpleFileProviderBackend</value>
      </property>

    <property>
        <name>sentry.hive.provider.resource</name>
        <value>file:///opt/mapr/sentry/sentry-1.4.0/conf/global-policy.ini</value>
    </property>

Step 3) Please add below property in global-policy.ini

[root@mfs021 ~]# vim /opt/mapr/sentry/sentry-1.4.0/conf/global-policy.ini

[groups]
mapr = admin_role
sentry_user = user_role

[roles]
admin_role = server=HS2
user_role = server=HS2->db=default->table=*->action=Select


Step 4) Add user and group

groupadd sentry_user
useradd -G sentry_user sentry_user1


Stpe 5) Check you configuration




[mapr@maprdemo ~]$ /opt/mapr/hive/hive-0.13/bin/beeline
beeline> !connect jdbc:hive2://localhost:10000
scan complete in 4ms
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: sentry_user1
Enter password for jdbc:hive2://localhost:10000: *
Connected to: Apache Hive (version 0.13.0-mapr-1510)
Driver: Hive JDBC (version 0.13.0-mapr-1510)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> show tables;
+-----------+
| tab_name  |
+-----------+
+-----------+
No rows selected (0.409 seconds)
0: jdbc:hive2://localhost:10000> create table xyz(id int);
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 Required privileges for this query: Server=HS2->Db=default->action=*; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000> show tables;
+-----------+
| tab_name  |
+-----------+
| mno       |
+-----------+
1 row selected (0.346 seconds)
0: jdbc:hive2://localhost:10000> drop table mno;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 Required privileges for this query: Server=HS2->Db=default->Table=mno->action=*; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000> select * from mno;
+---------+
| mno.id  |
+---------+
+---------+
No rows selected (0.616 seconds)
0: jdbc:hive2://localhost:10000>


Ansible Cheat sheet

Install Ansible  # yum install ansible Host file configuration  File  [ansible@kuber2 ~]$ cat /etc/ansible/hosts     [loca...