构建高可用服务器之四 Keepalive冗余Nginx
实验目的：
使用Keepalive来保证Nginx的高可用，进而实现网站的高可用！
实验拓扑：
实验环境：
如上拓扑图所示，master与slave是mysql主从，且有discuz论坛，且已经做高可用。(可参加《构建高可用服务器之三Keepalive冗余Mysql》)。现在是刚搭建两台Nginx,均已安装keepalive，IP地址分别为192.168.1.2和192.168.1.5(可参加《构建Nginx服务器之一安装及虚拟主机配置》和《构建高可用服务器之一Keepalive介绍及安装》)。
一、配置主机Nginx(192.168.1.2)
nginx服务的配置文件如下：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
|
[[email protected] ~]# cat/usr/local/nginx/conf/nginx.conf
worker_processes 1; events { worker_connections 1024;
} http { include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
###############################################################################upstream discuz { server 192.168.1.3:80 weight=1max_fails=2 fail_timeout=30s;
server 192.168.1.4:80 weight=1max_fails=2 fail_timeout=30s;
}
server {
listen 80;
server_name localhost;
location/ {
root html;
index index.html index.htm;
proxy_set_header Host $host;
proxy_set_header X-Real-IP$remote_addr;
proxy_set_headerX-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://discuz;
}
################################动静分离####################################### location~ .*\.(php|jsp|cgi|shtml)?$
{
proxy_set_header Host $host;
proxy_set_header X-Real-IP$remote_addr;
proxy_set_headerX-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://discuz;
}
location ~ .*\.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$
{
root/usr/local/nginx/html;
expires 30d;
}
####################################################################### error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}[[email protected] ~]#
|
keepaive配置文件如下：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
|
[[email protected] ~]# cat/etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs { notification_email {
}
notification_email_from [email protected]
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
} vrrp_script chk_nginx { script"/data/sh/check_nginx.sh"
interval2
weight2
} vrrp_instance VI_1 { state BACKUP
interface eth0
virtual_router_id 251
priority 100
mcast_src_ip 192.168.1.2
advert_int 1
nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.88
}
track_script{
chk_nginx
}
} |
主机nginx上keepalive脚本文件：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
[[email protected] ~]# mkdir -p /data/sh
[[email protected] ~]# touch check_nginx.sh
[[email protected] ~]# chmod +x check_nginx.sh
[[email protected] ~]# vim check_nginx.sh
[[email protected] ~]# mv check_nginx.sh /data/sh/
[[email protected] ~]# cat /data/sh/check_nginx.sh
#!/bin/bash#Target: If Nginx Down, Stop keepalive!#killall -0 nginx if [[ ! $? -eq 0 ]];then
/etc/init.d/keepalived stop
fi[[email protected] ~]#
|
加载模块
|
1
2
3
|
[[email protected] ~]# modprobe ip_vs
[[email protected] ~]# modprobe ip_vs_rr
[[email protected] ~]# modprobe ip_vs_wrr
|
启动服务
|
1
2
3
4
5
|
[[email protected] ~]# /etc/init.d/keepalivedrestart
Stopping keepalived: [FAILED]Starting keepalived: [ OK ][[email protected] ~]# /usr/local/nginx/sbin/nginx
[[email protected] ~]#
|
二、配置主机nginx2(192.168.1.5)
关于nginx服务配置文件：
与nginx主机(192.168.1.2完全一致)
关于keepalive
只需要把配置文件keepalived.conf中的mcast_src_ip 192.168.1.2改为mcast_src_ip192.168.1.5
关于keepalived的脚步配置文件
与nginx主机(192.168.1.2完全一致)
启动nginx与keepalive服务
三、测试
可以通过Nginx上的VIP 192.168.1.88访问
停止nginx主机上的nginx服务，VIP自动漂移到nginx2上，论坛已经可以访问;
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
[[email protected] ~]# ps -ef | grep nginx
root 2067 1 0 21:31 ? 00:00:00 nginx: master process/usr/local/nginx/sbin/nginx
nobody 2070 2067 0 21:31 ? 00:00:00 nginx: worker process root 6302 1946 0 22:24 pts/0 00:00:00 grep nginx
[[email protected] ~]# /etc/init.d/keepalivedstatus
keepalived (pid 4430) is running...[[email protected] ~]# pkill nginx
[[email protected] ~]# /etc/init.d/keepalivedstatus
keepalived is stopped[[email protected] ~]# tail /var/log/messages
May 21 22:18:25 nginx NetworkManager[1517]:<info> prefix 24 (255.255.255.0)May 21 22:18:25 nginx NetworkManager[1517]:<info> gateway 192.168.33.2May 21 22:18:25 nginx NetworkManager[1517]:<info> nameserver '192.168.33.2'
May 21 22:18:25 nginx NetworkManager[1517]:<info> domain name 'localdomain'
May 21 22:24:20 nginx Keepalived:Terminating on signalMay 21 22:24:20 nginx Keepalived_vrrp:Terminating VRRP child process on signalMay 21 22:24:20 nginx Keepalived_vrrp:VRRP_Instance(VI_1) removing protocol VIPs.May 21 22:24:20 nginxKeepalived_healthcheckers: Netlink reflector reports IP 192.168.1.88 removedMay 21 22:24:20 nginxKeepalived_healthcheckers: Terminating Healthchecker child process on signalMay 21 22:24:20 nginx Keepalived: StoppingKeepalived v1.1.15 (05/21,2015)
[[email protected] ~]#
|
此时VIP在nginx2主机上(192.168.1.5)
论坛依旧可以访问，Nginx高可用实现成功。
四、总结
1、 selinux与iptables注意要配置或关闭及清空。
2、 注意加载ip_vs、ip_vs_rr、ip_vs_wrr模块.(根据需要加载即可)。
3、 Nginx做的有动静分离，那么两台nginx上的网站发布目录要和后台apache网站发布目录内容完全一致。
4、编译安装keepalive的时候，指定的内核一定要与本机内核一致。
5、同一组keepalive的virtual_router_id必须一致，同一局域网内有多组keepalive的时候，virtual_router_id不能一致。
（这个拓扑图中，Mysql那一组keepalive的virtual_router_id一致均为51，Nginx那一组keepalive的virtual_router_id一致均为251，因为mysql与nginx在同一局域网内，所以这两组keepalive的virtual_router_id一定不能一样。）
6、 当keepalive中开启nopreempt的时候，两台keepalive的state均为BACKUP
(nopreempt这个选项为非抢占，忽略优先级的存在，当nginx主机宕机的时候，VIP会飘到nginx2主机上，并且即使nginx主机恢复正常，VIP依旧指向nginx2，这样可以减少短暂服务中断的次数，2次变1次)