How To Install and configure Elasticsearch, Logstash, and Kibana (ELK Stack)

1) Add  ELK repo



[root@cdh084 ~]# cat /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-2.x]
name=Elasticsearch repository for 2.x packages
baseurl=http://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

[root@cdh084 ~]# cat /etc/yum.repos.d/logstash.repo
[logstash-2.2]
name=logstash repository for 2.2 packages
baseurl=http://packages.elasticsearch.org/logstash/2.2/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1

[root@cdh084 ~]# cat /etc/yum.repos.d/kibana.repo
[kibana-4.4]
name=Kibana repository for 4.4.x packages
baseurl=http://packages.elastic.co/kibana/4.4/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1
[root@cdh084 ~]



A) Install Logstash

1) Install logstash

[root@cdh081 ~]# yum install -y logstash

2) On logstash installed host add below files

[root@cdh081 ~]# cat  /etc/logstash/conf.d/logstash.conf

input {
      file {
          path => "/var/log/hadoop-hdfs/hadoop-hdfs-*-cdh08*.tuxhub.com.log"
          start_position => "beginning"
      }
}

filter {

}

output {
    elasticsearch {
        action => "index"
        hosts => ["cdh084:9200"]
        index => "logstash-%{+YYYY.MM.dd}"
        workers => 1
    }
}

[root@cdh081 ~]#

Elasticsearch :

1) Install Elasticsearch :

[root@cdh081 ~]# sudo yum -y install elasticsearch

2) Add below line in elasticsearch conf

[root@cdh084 ~]# cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^#|^$"
network.host: cdh084
http.port: 9200
cluster.name: "logsearch"
node.master: true
node.data: true
index.number_of_shards: 5
index.number_of_replicas : 1
path.data: /home/es/
[root@cdh084 ~]#

3) Install Kibana

[root@cdh084 ~]#  sudo yum -y install kibana


Restart Services

1) Logstash :

[root@cdh084 ~]#  /etc/init.d/logstash restart

2) elasticsearch

[root@cdh084 ~]#  /etc/init.d/elasticsearch restart

3) Kibana

[root@cdh084 ~]#  /etc/init.d/kibana restart

WEBUI :- http://cdh084:5601/