ELK技术用于收集分析缓存数据库Redis的数据
拓扑结构：
Redis------>Logstash------->Elasticsearch<-----------------Kibana
1. redis (192.168.1.115)
[[email protected] ~]# redis-server /etc/redis.conf
22575:C 23 Mar 17:22:07.795 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
22575:C 23 Mar 17:22:07.795 # Redis version=4.0.6, bits=64, commit=00000000, modified=0, pid=22575, just started
22575:C 23 Mar 17:22:07.795 # Configuration loaded
[[email protected] ~]# netstat -tulnp | grep 63
tcp 0 0 192.168.1.115:6379 0.0.0.0:* LISTEN 22576/redis-server
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 22576/redis-server
[[email protected] ~]# redis-cli
127.0.0.1:6379> ping
PONG
2. logstash(192.168.1.112)
定义redis中的数据类型，键，端口，数据库编号
[[email protected] conf.d]# vim redis1.conf
input {
redis {
data_type => "list"
key => "logstash-test-list"
host => "192.168.1.115"
port => 6379
db => 0
}
}
output {
stdout{
codec => "rubydebug"
}
elasticsearch{
hosts => ["192.168.1.113"]
index => "messages-%{+YYYY.MM.dd}"
}
}
3. elasticsearch (192.168.1.113)
[[email protected] ~]# netstat -tunlp | grep 92
tcp6 0 0 192.168.1.113:9200 :::* LISTEN 4149/java
tcp6 0 0 192.168.1.113:9300 :::* LISTEN 4149/java
4. kibana (192.168.1.114)
[[email protected] ~]# netstat -tunlp | grep 56
tcp 0 0 192.168.1.114:5601 0.0.0.0:* LISTEN 6227/node
5. logstash (192.168.1.112)
[[email protected] conf.d]# /usr/share/logstash/bin/logstash -f redis1.conf
在redis端输入：
127.0.0.1:6379> LPUSH logstash-test-list 02
(integer) 1
127.0.0.1:6379> LPUSH logstash-test-list 02
(integer) 1
127.0.0.1:6379> LPUSH logstash-test-list 03
(integer) 1
logstash（192.168.1.112）端显示：
[ERROR] 2019-03-23 17:42:43.983 [[main]<redis] json - JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Invalid numeric value: Leading zeroes not allowed
at [Source: (String)"02"; line: 1, column: 2]>, :data=>"02"}
{
"@timestamp" => 2019-03-23T09:42:43.991Z,
"tags" => [
[0] "_jsonparsefailure"
],
"@version" => "1",
"message" => "02"
}
[ERROR] 2019-03-23 17:42:49.857 [[main]<redis] json - JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Invalid numeric value: Leading zeroes not allowed
at [Source: (String)"02"; line: 1, column: 2]>, :data=>"02"}
{
"@timestamp" => 2019-03-23T09:42:49.857Z,
"tags" => [
[0] "_jsonparsefailure"
],
"@version" => "1",
"message" => "02"
}
[ERROR] 2019-03-23 17:42:52.390 [[main]<redis] json - JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Invalid numeric value: Leading zeroes not allowed
at [Source: (String)"03"; line: 1, column: 2]>, :data=>"03"}
{
"@timestamp" => 2019-03-23T09:42:52.391Z,
"tags" => [
[0] "_jsonparsefailure"
],
"@version" => "1",
"message" => "03"
}
kibana(192.168.1.114)端显示：