rsyslog与loganalyzer
1、rsyslog日志服务的概述
rsyslog主要记录linux服务器上各发生的事件日志信息的，是多线程工作方式，可以通过TCP，UDP工作，也支持SSL，TLS，RELP；也支持把日志信息存储于数据库中，如MySQL，PGSQL，Oracle来实现日志存储。
syslogd —> 主要记录登录系统与网络等服务的信息
klogd —> 主要记录内核产生的各项信息
logrotate —> 主要进行日志文件的轮替功能
日志收集方：
facility ：设施，从功能或程序上对日志进行分类；
auth（认证授权相关），authpriv，cron，daemon，kern，lpr（打印系统），mail，mark，news，security，user，uucp，local0-local7（用户可自定义的类别），syslog
priority
debug：调试级别；
info：仅仅是一些基本的信息说明而已；
notice：除了info外还需要注意的一些信息内容；
warn：警示的信息，可能有问题，但还不至于影响daemon的运行；
error：一些重大的错误信息，如配置文件的某些设置值造成该服务无法启动的信息说明；
crit：比error更严重的错误信息，到达临界点，已经非常严重；
alert：警告，已经很有问题的等级；
emerg：“疼痛级别”，指系统已经几乎要死机的状态了，很严重的错误信息，通常只有硬件出问题导致整个内核无法运行才会出现的等级；
*指定级别：
：所有级别
none：没有级别
priority：此级别及更高级别的日志信息
=priority：此级别
syslogd的主配置文件：
主程序：rsyslogd
配置文件：/etc/rsyslog.conf
服务脚本：/etc/rc.d/init.d/rsyslog
文件记录的日志的格式：
事件产生的日期时间 主机 进程（pid）：事件内容
有些日志记录二进制格式：/var/log/wtmp，/var/log/btmp
/var/log/wtmp：当前系统上成功登录的日志；
last
/var/log/btmp：当前系统上失败的登录尝试；
lastb
lastlog命令：显示当前系统每一个用户最近一次的登录时间；
2、自定义日志存放路径
1）修改sshd配置文件的facility
# Logging
# obsoletes QuietMode and FascistLogging
#SyslogFacility AUTH
SyslogFacility local2 //只需要把此处的类型改为local2用户自定即可
#LogLevel INFO
-- INSERT --
2）编辑rsyslog的主配置文件
[[email protected] ~]# vim /etc/rsyslog.conf
# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514 //开启udp的服务端口
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514 //开启tcp的服务端口
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
local2.* /var/log/sshd.log ##在这里添加一个自定义的日志记录信息
3）重启sshd服务和rsyslog服务
[[email protected] ~]# systemctl reload sshd
[[email protected] ~]# systemctl restart rsyslog
[[email protected] ~]# netstat -tunlp | grep 514
tcp 0 0 0.0.0.0:514 0.0.0.0:* LISTEN 2744/rsyslogd
tcp6 0 0 :::514 :::* LISTEN 2744/rsyslogd
udp 0 0 0.0.0.0:514 0.0.0.0:* 2744/rsyslogd
udp6 0 0 :::514 :::* 2744/rsyslogd
[[email protected] ~]#
4）开启一个新终端ssh连接，并查看日志信息
[[email protected] ~]# tail -f /var/log/sshd.log
Mar 21 15:04:18 master sshd[2783]: Accepted password for root from 192.168.126.1 port 12134 ssh2
//此时sshd服务相关的日志信息已经存放到自定义的日志文件目录
3、把本机日志存放于其他服务器上
1）修改rsyslog主配置文件
[[email protected] ~]#vim /etc/rsyslog.conf
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
#*.info;mail.none;authpriv.none;cron.none /var/log/messages
*.info;mail.none;authpriv.none;cron.none @192.168.126.129 //把当前类型的日志都发送给这个ip地址的服务器
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
2）编辑日志服务器的配置文件，开启端口
# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
3）重启客户端与日志服务器的rsyslog服务
[[email protected] yum.repos.d]# systemctl restart rsyslog
[[email protected] yum.repos.d]# netstat -tunlp | grep 514
tcp 0 0 0.0.0.0:514 0.0.0.0:* LISTEN 34493/rsyslogd
tcp6 0 0 :::514 :::* LISTEN 34493/rsyslogd
udp 0 0 0.0.0.0:514 0.0.0.0:* 34493/rsyslogd
udp6 0 0 :::514 :::* 34493/rsyslogd
[[email protected] yum.repos.d]#
[[email protected] yum.repos.d]#
4）在服务器端安装一个软件包测试
[[email protected] ~]# yum install vsftpd -y
Loaded plugins: langpacks, product-id, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Resolving Dependencies
--> Running transaction check
---> Package vsftpd.x86_64 0:3.0.2-9.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
======================================================================================================================
Package Arch Version Repository Size
======================================================================================================================
Installing:
vsftpd x86_64 3.0.2-9.el7 jyy 166 k
Transaction Summary
======================================================================================================================
Install 1 Package
Total download size: 166 k
Installed size: 343 k
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : vsftpd-3.0.2-9.el7.x86_64 1/1
Verifying : vsftpd-3.0.2-9.el7.x86_64 1/1
Installed:
vsftpd.x86_64 0:3.0.2-9.el7
Complete!
5）在日志服务器上查看是否有此记录
[[email protected] ~]# cat /var/log/messages | grep vsftpd
Mar 21 15:17:32 master yum[3073]: Installed: vsftpd-3.0.2-9.el7.x86_64
//已经有此条主机的记录信息
4、把客户端日志存放在数据库中
1）在服务器端安装数据库并登录
[[email protected] ~]# yum install mariadb mariadb-server -y
[[email protected] ~]# systemctl enable mariadb
ln -s '/usr/lib/systemd/system/mariadb.service' '/etc/systemd/system/multi-user.target.wants/mariadb.service'
[[email protected] ~]# systemctl restart mariadb
[[email protected] ~]# mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 5.5.60-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]>
2）在客户端安装支持连接mysql的软件包
[[email protected] ~]# yum list all | grep rsyslog-mysql
rsyslog-mysql.x86_64 7.4.7-7.el7_0 jyy
[[email protected] ~]# yum install rsyslog-mysql -y
Loaded plugins: langpacks, product-id, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Resolving Dependencies
--> Running transaction check
---> Package rsyslog-mysql.x86_64 0:7.4.7-7.el7_0 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
======================================================================================================================
Package Arch Version Repository Size
======================================================================================================================
Installing:
rsyslog-mysql x86_64 7.4.7-7.el7_0 jyy 29 k
Transaction Summary
======================================================================================================================
Install 1 Package
Total download size: 29 k
Installed size: 16 k
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : rsyslog-mysql-7.4.7-7.el7_0.x86_64 1/1
Verifying : rsyslog-mysql-7.4.7-7.el7_0.x86_64 1/1
Installed:
rsyslog-mysql.x86_64 0:7.4.7-7.el7_0
Complete!
[[email protected] ~]#
[[email protected] ~]# rpm -ql rsyslog-mysql
/usr/lib64/rsyslog/ommysql.so //这个是支持mysql连接的模块
/usr/share/doc/rsyslog-mysql-7.4.7
/usr/share/doc/rsyslog-mysql-7.4.7/createDB.sql //这个是创建记录数据库的文件
[[email protected] ~]#
3）在服务器端创建一个授权登录用户
MariaDB [(none)]> grant all on Syslog.* to 'syslog'@'192.168.126.%' identified by 'syslog';
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]>
4）编辑mysql的主配置文件
[[email protected] ~]# vim /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
server-id=3
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
###添加以下两行内容，跳过解析
skip_name_resolve = on
innodb_file_per_table =on
5）重启数据库并测试账号
[[email protected] ~]# systemctl restart mariadb
[[email protected] ~]# mysql -usyslog -psyslog -h192.168.126.129 //注意：主机要写ip具体的地址，不然服务器会用默认的127.0.0.1
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 5.5.60-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]>
6）在客户端导入建库语句并查看
[[email protected] rsyslog-mysql-7.4.7]# mysql -usyslog -p -h192.168.126.129 < /usr/share/doc/rsyslog-mysql-7.4.7/createDB.sql
Enter password:
[[email protected] rsyslog-mysql-7.4.7]#
在服务器端查看是否有库表
[[email protected] ~]# mysql -usyslog -psyslog -h192.168.126.129
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 16
Server version: 5.5.60-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| Syslog |
| test |
+--------------------+
3 rows in set (0.00 sec)
MariaDB [(none)]> use Syslog;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MariaDB [Syslog]> show tables;
+------------------------+
| Tables_in_Syslog |
+------------------------+
| SystemEvents |
| SystemEventsProperties |
+------------------------+
2 rows in set (0.00 sec)
MariaDB [Syslog]>
7）编辑客户端rsyslog配置文件，添加模块功能和认证
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
$ModLoad ommysql //添加此模块
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
#*.info;mail.none;authpriv.none;cron.none /var/log/messages
*.info;mail.none;authpriv.none;cron.none :ommysql:192.168.126.129,Syslog,syslog,syslog
//注明使用的模块，服务器IP地址，库名，用户名，密码
8）重启服务并测试
[[email protected] ~]# systemctl restart rsyslog
[[email protected] ~]# yum install vsftpd -y
Loaded plugins: langpacks, product-id, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Resolving Dependencies
--> Running transaction check
---> Package vsftpd.x86_64 0:3.0.2-9.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
======================================================================================================================
Package Arch Version Repository Size
======================================================================================================================
Installing:
vsftpd x86_64 3.0.2-9.el7 jyy 166 k
Transaction Summary
====================================================================================================================
在服务器端查看是否有此条记录
MariaDB [Syslog]> select * from SystemEvents\G;
*************************** 14. row ***************************
ID: 14
CustomerID: NULL
ReceivedAt: 2019-03-21 16:11:52
DeviceReportedTime: 2019-03-21 16:11:52
Facility: 1
Priority: 6
FromHost: master
Message: Installed: vsftpd-3.0.2-9.el7.x86_64
NTSeverity: NULL
Importance: NULL
EventSource: NULL
EventUser: NULL
EventCategory: NULL
EventID: NULL
EventBinaryData: NULL
MaxAvailable: NULL
CurrUsage: NULL
MinUsage: NULL
MaxUsage: NULL
InfoUnitID: 1
SysLogTag: yum[33697]:
EventLogType: NULL
GenericFileName: NULL
SystemID: NULL
5、配置loganalyzer日志管理界面
1）在客户端安装软件包
[[email protected] ~]# yum install httpd php php-mysql -y
Loaded plugins: langpacks, product-id, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Package httpd-2.4.6-31.el7.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package php.x86_64 0:5.4.16-23.el7_0.3 will be installed
--> Processing Dependency: php-common(x86-64) = 5.4.16-23.el7_0.3 for package: php-5.4.16-23.el7_0.3.x86_64
--> Processing Dependency: php-cli(x86-64) = 5.4.16-23.el7_0.3 for package: php-5.4.16-23.el7_0.3.x86_64
---> Package php-mysql.x86_64 0:5.4.16-23.el7_0.3 will be installed
--> Processing Dependency: php-pdo(x86-64) = 5.4.16-23.el7_0.3 for package: php-mysql-5.4.16-23.el7_0.3.x86_64
--> Running transaction check
---> Package php-cli.x86_64 0:5.4.16-23.el7_0.3 will be installed
---> Package php-common.x86_64 0:5.4.16-23.el7_0.3 will be installed
--> Processing Dependency: libzip.so.2()(64bit) for package: php-common-5.4.16-23.el7_0.3.x86_64
---> Package php-pdo.x86_64 0:5.4.16-23.el7_0.3 will be installed
--> Running transaction check
---> Package libzip.x86_64 0:0.10.1-8.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
======================================================================================================================
Package Arch Version Repository Size
======================================================================================================================
Installing:
php x86_64 5.4.16-23.el7_0.3 jyy 1.3 M
php-mysql x86_64 5.4.16-23.el7_0.3 jyy 97 k
Installing for dependencies:
libzip x86_64 0.10.1-8.el7 jyy 49 k
php-cli x86_64 5.4.16-23.el7_0.3 jyy 2.7 M
php-common x86_64 5.4.16-23.el7_0.3 jyy 561 k
php-pdo x86_64 5.4.16-23.el7_0.3 jyy 95 k
Transaction Summary
======================================================================================================================
Install 2 Packages (+4 Dependent packages)
Total download size: 4.9 M
Installed size: 18 M
Downloading packages:
----------------------------------------------------------------------------------------------------------------------
Total 24 MB/s | 4.9 MB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libzip-0.10.1-8.el7.x86_64 1/6
Installing : php-common-5.4.16-23.el7_0.3.x86_64 2/6
Installing : php-pdo-5.4.16-23.el7_0.3.x86_64 3/6
Installing : php-cli-5.4.16-23.el7_0.3.x86_64 4/6
Installing : php-5.4.16-23.el7_0.3.x86_64 5/6
Installing : php-mysql-5.4.16-23.el7_0.3.x86_64 6/6
Verifying : php-common-5.4.16-23.el7_0.3.x86_64 1/6
Verifying : php-pdo-5.4.16-23.el7_0.3.x86_64 2/6
Verifying : php-5.4.16-23.el7_0.3.x86_64 3/6
Verifying : libzip-0.10.1-8.el7.x86_64 4/6
Verifying : php-mysql-5.4.16-23.el7_0.3.x86_64 5/6
Verifying : php-cli-5.4.16-23.el7_0.3.x86_64 6/6
Installed:
php.x86_64 0:5.4.16-23.el7_0.3 php-mysql.x86_64 0:5.4.16-23.el7_0.3
Dependency Installed:
libzip.x86_64 0:0.10.1-8.el7 php-cli.x86_64 0:5.4.16-23.el7_0.3 php-common.x86_64 0:5.4.16-23.el7_0.3
php-pdo.x86_64 0:5.4.16-23.el7_0.3
Complete!
[[email protected] ~]#
2）编辑是否可以连接MySQL的测试页面
[[email protected] ~]#
[[email protected] ~]# vim /var/www/html/index.php
<?php
$conn = mysql_connect('192.168.126.129','syslog','syslog');
if ($conn)
echo "Ok";
else
echo "Failure";
?>
3）下载loganalyzer软件包
[[email protected] ~]# ls
all_bak.sql binlog.sql Documents loganalyzer-3.6.5.tar.gz Public
anaconda-ks.cfg CentOS-Base.repo Downloads Music Templates
apache-tomcat-8.5.37.tar.gz Desktop initial-setup-ks.cfg Pictures Videos
[[email protected] ~]# tar xf loganalyzer-3.6.5.tar.gz
[[email protected] ~]# mv loganalyzer-3.6.5 /var/www/html/log
[[email protected] ~]# cd /var/www/html/
[[email protected] html]# ls
index.php log
[[email protected] html]# ls
index.php log
[[email protected] html]# mv log/src loganalyzer
[[email protected] html]# ls
index.php log loganalyzer
[[email protected] html]# cd log
[[email protected] log]# ls
ChangeLog contrib COPYING doc INSTALL
[[email protected] log]# cp contrib/* ../loganalyzer/
[[email protected] log]# cd ..
[[email protected] html]# ls
index.php log loganalyzer
[[email protected] html]# cd loganalyzer/
[[email protected] loganalyzer]# chmod +x *.sh
[[email protected] loganalyzer]#
[[email protected] loganalyzer]#
[[email protected] loganalyzer]# touch config.php
[[email protected] loganalyzer]# chmod 666 config.php
[[email protected] loganalyzer]# ll
total 240
drwxrwxr-x 2 root root 4096 Oct 9 2013 admin
-rw-rw-r-- 1 root root 5509 Oct 9 2013 asktheoracle.php
drwxrwxr-x 2 root root 4096 Oct 9 2013 BitstreamVeraFonts
-rw-rw-r-- 1 root root 18492 Oct 9 2013 chartgenerator.php
drwxrwxr-x 6 root root 4096 Oct 9 2013 classes
-rw-rw-rw- 1 root root 0 Mar 21 16:49 config.php
-rwxr-xr-x 1 root root 49 Mar 21 16:47 configure.sh
-rw-rw-r-- 1 root root 9205 Oct 9 2013 convert.php
drwxrwxr-x 2 root root 110 Oct 9 2013 cron
drwxrwxr-x 2 root root 60 Oct 9 2013 css
-rw-rw-r-- 1 root root 16499 Oct 9 2013 details.php
drwxr-xr-x 2 root root 4096 Oct 9 2013 doc
-rw-rw-r-- 1 root root 12748 Oct 9 2013 export.php
-rw-rw-r-- 1 root root 1150 Oct 9 2013 favicon.ico
drwxrwxr-x 5 root root 60 Oct 9 2013 images
drwxrwxr-x 2 root root 4096 Oct 9 2013 include
-rw-rw-r-- 1 root root 38383 Oct 9 2013 index.php
-rw-rw-r-- 1 root root 41586 Oct 9 2013 install.php
drwxrwxr-x 2 root root 45 Oct 9 2013 js
drwxrwxr-x 4 root root 24 Oct 9 2013 lang
-rw-rw-r-- 1 root root 3395 Oct 9 2013 login.php
-rw-rw-r-- 1 root root 7164 Oct 9 2013 reportgenerator.php
-rw-rw-r-- 1 root root 4146 Oct 9 2013 reports.php
-rw-rw-r-- 1 root root 9438 Oct 9 2013 search.php
-rwxr-xr-x 1 root root 31 Mar 21 16:47 secure.sh
-rw-rw-r-- 1 root root 4878 Oct 9 2013 statistics.php
drwxrwxr-x 3 root root 4096 Oct 9 2013 templates
drwxrwxr-x 4 root root 31 Oct 9 2013 themes
-rw-rw-r-- 1 root root 3383 Oct 9 2013 userchange.php
[[email protected] loganalyzer]#
4)进入安装页面安装
指明数据库服务器ip、数据库名、用户名和密码