Elastic Stack 6.7 安装部署尝鲜，基于centos7.6最新系统。使用docker安装

Elastic Stack 6.7.0尝鲜

基于CentOS7.6, Docker-CE 18.09.4, Docker-web-Ui, Elasticsearch 6.7, Kibana 6.7, Logstash6.7,X-Pack

Akide_Liu

原文地址：Elastic Stack 6.7.0尝鲜 Powered By LLYCloud

安装顺序：

1. Elasticsearch (install
   instructions)主要搜索引擎

2. Kibana
   (install)图形化web界面

3. Logstash
   (install)log的汇总与收集（server端）

4. Beats (install
   instructions)log收集（client端）

5. Elasticsearch Hadoop (install
   instructions)（Hadoop大文件系统，本次不用。）

在安装手册上面有多重方法包括，源码，deb，rpm等等，在这里我们使用源码基于centos7.6.

• 官网推荐使用java版本Oracle JDK version 1.8.0_131

• 必须有一个专用的用户运行。

Reference：https://www.elastic.co/guide/en/elasticsearch/reference/6.7/docker.html

docker	Images are available for running Elasticsearch as Docker containers. They may be downloaded from the Elastic Docker Registry. Install Elasticsearch with Docker

Docker安装ELK：

Reference：https://elk-docker.readthedocs.io/

Docker hub：https://hub.docker.com/r/sebp/elk/

1. 环境准备

   1. 安装最新版docker-ce

https://docs.docker.com/install/linux/docker-ce/centos/

卸载老版本：

yum remove docker \\

docker-client \\

docker-client-latest \\

docker-common \\

docker-latest \\

docker-latest-logrotate \\

docker-logrotate \\

docker-engine

安装docker-ce依赖

sudo yum install -y yum-utils \\

device-mapper-persistent-data \\

lvm2

添加yum repo

sudo yum-config-manager \\

\--add-repo \\

https://download.docker.com/linux/centos/docker-ce.repo

启用repo

\$ sudo yum-config-manager --enable docker-ce-nightly

开始安装docker-ce

sudo yum install docker-ce docker-ce-cli containerd.io

启动docker服务并开机自启动：

sudo systemctl start docker

sudo systemctl enable docker

可以安装一个docker-ui（对于不太会用docker的，比如我）

https://hub.docker.com/r/uifd/ui-for-docker/

复制粘贴：

Quickstart

1. Run: docker run -d -p 9000:9000 --privileged -v
   /var/run/docker.sock:/var/run/docker.sock uifd/ui-for-docker

2. Open your browser to http://<dockerd host ip>:9000

   1. 最小内存要求4GB for docker

   2. Mmap
      counts高于262,144（不太清楚具体是啥，应该是虚拟内存类似的，有兴趣自己看原文。）

sysctl -w vm.max_map_count=262144

1. 设置防火墙允许相关端口通讯，至少保证5044.推荐关闭防火墙（内网）

systemctl stop firewalld

systemctl status firewalld

● firewalld.service - firewalld - dynamic firewall daemon

Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor
preset: enabled)

Active: inactive (dead)

Docs: man:firewalld(1)

1. 下载ELK-docker

docker pull sebp/elk

当然可以选取所需要的版本：

For instance, the image containing Elasticsearch 1.7.3, Logstash 1.5.5, and
Kibana 4.1.2 (which is the last image using the Elasticsearch 1.x and Logstash
1.x branches) bears the tag E1L1K4, and can therefore be pulled using sudo
docker pull sebp/elk:E1L1K4.

现在的least版本也就是default版本是ELK 6.7

1. 启动docker of ELK

sudo docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -p 9300:9300 -it --name
elk sebp/elk &

想看安装过程的可以去ui-docker里面看，但是如果不加&后面log输出太多会导致一旦中断命令就会关闭ELK
Stack。

1. 使用的端口：

   1. 5601 Kibana web界面

   2. 9200 Elasticsearch JSON 接口

   3. 5044 Logstash Beats 接口

1. Access Kibana’s web interface by browsing to http://<your-host>:5601,
   where <your-host> is the hostname or IP address of the host Docker is
   running on (see note), e.g. localhost if running a local native version of
   Docker, or the IP address of the virtual machine if running a VM-hosted
   version of Docker (see note).

（访问http://主机名:5601到Kibana web）

首先需要打开x-pack（安全模块，才可以设置密码。）

打开kibana —> License Management —> Start a 30-day trial

1. 进入ELK-docker配置默认用户的密码：

Reference：https://www.elastic.co/guide/en/elasticsearch/reference/6.7/configuring-security.html

docker exec -it elk /bin/bash

/opt/elasticsearch/bin/elasticsearch-setup-passwords interactive

Unexpected response code [500] from calling GET
http://172.17.0.3:9200/_xpack/security/_authenticate?pretty

It doesn’t look like the X-Pack security feature is enabled on this
Elasticsearch node.

Please check if you have enabled X-Pack security in your elasticsearch.yml
configuration file.

ERROR: X-Pack Security is disabled by configuration.

设置密码会出现一个这个问题，提示X-pack需要enable

[email protected]:/# curl http://127.0.0.1:9200/_cat/health

1554113235 10:07:15 elasticsearch yellow 1 1 12 12 0 0 4 0 - 75.0%

[email protected]:/# curl
http://172.17.0.3:9200/_xpack/security/_authenticate?pretty

{

“error” : {

“root_cause” : [

{

“type” : “exception”,

“reason” : “Security must be explicitly enabled when using a trial license.
Enable security by setting [xpack.security.enabled] to [true] in the
elasticsearch.yml file and restart the node.”

}

],

“type” : “exception”,

“reason” : “Security must be explicitly enabled when using a trial license.
Enable security by setting [xpack.security.enabled] to [true] in the
elasticsearch.yml file and restart the node.”

},

“status” : 500

}

那么我们去到elasticsearch.yml添加：

[xpack.security.enabled] to [true]

vim /etc/elasticsearch/elasticsearch.yml

然后需要重新启动docker以重启服务：

[[email protected] conf.d]# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

c33dd5a85ae8 sebp/elk “/usr/local/bin/star…” 14 minutes ago Up 11 minutes
0.0.0.0:5044->5044/tcp, 0.0.0.0:5601->5601/tcp, 0.0.0.0:9200->9200/tcp,
0.0.0.0:9300->9300/tcp elk

6552e53ec2ee uifd/ui-for-docker “/ui-for-docker” 2 hours ago Up 2 hours
0.0.0.0:9000->9000/tcp stupefied_mclaren

[[email protected] conf.d]# docker restart c33dd5a85ae8

c33dd5a85ae8

然后我们重新进入docker-shell进行设置初始密码：

/opt/elasticsearch/bin/elasticsearch-setup-passwords interactive

Initiating the setup of passwords for reserved users
elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.

You will be prompted to enter passwords as the process progresses.

Please confirm that you would like to continue [y/N]y

Enter password for [elastic]:

Reenter password for [elastic]:

Enter password for [apm_system]:

Reenter password for [apm_system]:

Enter password for [kibana]:

Reenter password for [kibana]:

Enter password for [logstash_system]:

Reenter password for [logstash_system]:

Enter password for [beats_system]:

Reenter password for [beats_system]:

Enter password for [remote_monitoring_user]:

Reenter password for [remote_monitoring_user]:

Changed password for user [apm_system]

Changed password for user [kibana]

Changed password for user [logstash_system]

Changed password for user [beats_system]

Changed password for user [remote_monitoring_user]

Changed password for user [elastic]

在这里设置了很多的用户，关于相关用于的专属功能原文：

https://www.elastic.co/guide/en/elastic-stack-overview/6.7/built-in-users.html

1. 因为启用了x-pack，所以相关组件需要设置用户名密码才能登陆。我们需要设置原来能够使用kibana,
   logstash

[email protected]:/# vim /opt/kibana/config/kibana.yml

#elasticsearch.username: “elastic”

#elasticsearch.password: “yourpass”

[email protected]:/# /etc/init.d/kibana restart

* Stopping Kibana5 [ OK ]

* Starting Kibana5 [ OK ]

[email protected]:/# vim /etc/logstash/conf.d/30-output.conf

#user => “elastic”

#password => “yourpassword”

（#要去掉）

[email protected]:/# vim /etc/logstash/conf.d/02-beats-input.conf

#ssl => true

#ssl_certificate => “/etc/pki/tls/certs/logstash-beats.crt”

#ssl_key => “/etc/pki/tls/private/logstash-beats.key”

（我这里去掉了ssl设置，因为是纯粹内网环境，安全完全没有问题）

[email protected]:/# /etc/init.d/logstash restart

Killing logstash (pid 368) with SIGTERM

Waiting for logstash (pid 368) to die…

Waiting for logstash (pid 368) to die…

logstash stop failed; still running.

logstash started.

（如果不行的话手动kill process然后使用启动脚本）

Log位置：

tail -f /var/log/logstash/logstash-plain.log

1. 使用logging添加系统的system logs

点开就有怎么安装的流程，但是注意这个是直接连接elasticsearch，我们要先连接elasticsearch来load
visualize和dashboard。

1. Download and install Filebeat

curl -L -O
https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.7.0-x86_64.rpm
sudo

rpm -vi filebeat-6.7.0-x86_64.rpm

1. Edit the configuration

Modify filebeat.yml to set the connection information:

output.elasticsearch:

hosts: ["<es_url>"]

username: “elastic”

password: “<password>”

setup.kibana:

host: “<kibana_url>”

1. Enable and configure the system module

sudo filebeat modules enable system

1. Start Filebeat

sudo filebeat setup

sudo service filebeat start

[[email protected] ~]# sudo filebeat setup

Loaded index template

Loading dashboards (Kibana must be running and reachable)

Loaded dashboards

Loaded machine learning job configurations

[[email protected] ~]# service metricbeat start

Redirecting to /bin/systemctl start metricbeat.service

Failed to start metricbeat.service: Unit not found.

[[email protected] ~]# service filebeat start

Starting filebeat (via systemctl): [ OK ]

1. output.logstash:

[[email protected] ~]# vim /etc/filebeat/filebeat.yml

#setup.kibana:

# Kibana Host

# Scheme and port can be left out and will be set to the default (http and
5601)

# In case you specify and additional path, the scheme is required:
http://localhost:5601/path

# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601

# host: “192.168.10.102:5601”

#output.elasticsearch:

# Array of hosts to connect to.

# hosts: [“192.168.10.102:9200”]

# Enabled ilm (beta) to use index lifecycle management instead daily indices.

#ilm.enabled: false

# Optional protocol and basic auth credentials.

#protocol: “https”

# username: “elastic”

# password: “yourpasswd”

output.logstash:

# The Logstash hosts

hosts: [“192.168.10.102:5044”]

（只这部分enable，剩下全部#）

1. 配置Metrics

   1. 下载安装

curl -L -O
https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-6.7.0-x86_64.rpm

sudo rpm -vi metricbeat-6.7.0-x86_64.rpm

1. 修改配置文件/etc/metricbeat/metricbeat.yml

output.elasticsearch:

hosts: ["<es_url>"]

username: “elastic”

password: “<password>”

setup.kibana:

host: “<kibana_url>”

1. 启动模块

sudo metricbeat modules enable system

1. 启动metricbeat

sudo metricbeat setup sudo service metricbeat start

1. 最终效果截图，我使用了Frp和nginx代理了kibana，https://elk.llycloud.com