Saturday, 5 December 2015

Impala through a Proxy for High Availability



Step 1
       
       Installing HAProxy

       [root@mfs021 ~]# yum install haproxy

Steps 2

      Edit Config file for haproxy

      [root@mfs021 ~]# cat /etc/haproxy/haproxy.cfg

#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
##---------------------------------------------------------------------

defaults
    mode                    tcp
    log                     global
    retries                 3
    maxconn                 3000
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend  main *:5000
    acl url_static       path_beg       -i /static /images /javascript /stylesheets
    acl url_static       path_end       -i .jpg .gif .png .css .js

    use_backend static          if url_static
    default_backend             impala

#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
backend static
    balance     roundrobin
    server      static 127.0.0.1:4331 check

#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------

backend impala
    mode tcp
    option tcplog
    balance leastconn

    server impala1 mfs022.tuxhub.com:21000
    server impala2 mfs023.tuxhub.com:21000

Step 3 :

       Connect to impala:

root@mfs021 ~]# impala-shell
Starting Impala Shell without Kerberos authentication
Error connecting: TTransportException, Could not connect to mfs021.tuxhub.com:21000
Welcome to the Impala shell. Press TAB twice to see a list of available commands.

Copyright (c) 2012 Cloudera, Inc. All rights reserved.

(Shell build version: Impala Shell v1.4.1 (2b626c8) built on Thu Feb  5 14:53:44 PST 2015)
[Not connected] > connect localhost:5000;
Connected to localhost:5000
Server version: impalad version 1.4.1 RELEASE (build 2b626c8e9f4c666d23872c228cf43daae4c9acbb)
[localhost:5000] > show tables;

     
  

1 comment:

  1. I have follow the instruction , but got this error

    [root@C90BFH01 ~]# impala-shell -i localhost:10001
    Starting Impala Shell without Kerberos authentication
    Error: Unable to communicate with impalad service. This service may not be an impalad instance. Check host:port and try again.
    Traceback (most recent call last):
    File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/bin/../lib/impala-shell/impala_shell.py", line 1173, in
    shell = ImpalaShell(options)
    File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/bin/../lib/impala-shell/impala_shell.py", line 172, in __init__
    self.do_connect(options.impalad)
    File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/bin/../lib/impala-shell/impala_shell.py", line 523, in do_connect
    self._connect()
    File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/bin/../lib/impala-shell/impala_shell.py", line 564, in _connect
    server_version = self.imp_client.connect()
    File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/impala-shell/lib/impala_client.py", line 232, in connect
    result = self.imp_service.PingImpalaService()
    File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/impala-shell/gen-py/ImpalaService/ImpalaService.py", line 223, in PingImpalaService
    return self.recv_PingImpalaService()
    File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/impala-shell/gen-py/ImpalaService/ImpalaService.py", line 238, in recv_PingImpalaService
    raise x
    thrift.Thrift.TApplicationException: Invalid method name: 'PingImpalaService'

    any helps ?

    ReplyDelete

Ansible Cheat sheet

Install Ansible  # yum install ansible Host file configuration  File  [ansible@kuber2 ~]$ cat /etc/ansible/hosts     [loca...