RHCS+GFS(红帽HA+GFS)

1.环境介绍

ha管理端：luci：192.168.122.1

ha节点ricci： 192.168.122.34 192.168.122.33

从desk82上使用iscsi方式共享磁盘共ha节点建立GFS文件系统


2.iscsi磁盘共享

服务器端：

采用LVM方式共享，以便后期的有可能的扩展；

[[email protected]~]# fdisk -cu /dev/sda


Command(m for help): n

Commandaction

e extended

p primary partition (1-4)

p

Selectedpartition 4

Lastsector, +sectors or +size{K,M,G} (11495424-16777215, default16777215): +2G

Command(m for help): T

Partitionnumber (1-4): 4

Hexcode (type L to list codes): 8e

Changedsystem type of partition 4 to 8e (Linux LVM)


Command(m for help): wq

[[email protected]~]# pvcreate /dev/sda5

[[email protected]~]# vgcreate ha /dev/sda5

[[email protected]~]# lvcreate -L 1900M -n hadamo ha

[[email protected]~]# yum -y install scsi-target-utils.x86_64


[[email protected]~]# vim /etc/tgt/targets.conf

<targetiqn.2013-09.desk82.example:server.target1>

backing-store /dev/ha/hadamo

initiator-address 192.168.122.33

initiator-address 192.168.122.34

</target>

[[email protected]~]# /etc/init.d/tgtd start

[[email protected]~]# tgtadm --lld iscsi --op show --mode target #检查共享是否成功

Target1: iqn.2013-09.desk82.example:server.target1

….........

Backing store path: /dev/ha/hadamo

Backing store flags:

Account information:

ACL information:

192.168.122.33

192.168.122.34


ha端导入硬盘：

分别在两个节点上进行：

[[email protected]~]# yum -y install iscsi-initiator-utils.x86_64

[[email protected]~]# iscsiadm -m discovery -t st -p 192.168.122.82

Startingiscsid: [ OK ]

192.168.122.82:3260,1iqn.2013-09.desk82.example:server.target1

[[email protected]~]# iscsiadm -m node -l iqn.2013-09.desk82.example:server.target1

[[email protected]~]# fdisk -l

…...........................

Disk/dev/sda: 1992 MB, 1992294400 bytes

62heads, 62 sectors/track, 1012 cylinders

Units =cylinders of 3844 * 512 = 1968128 bytes

…..............................

desk33同样操作



3。GFS文件系统的创建与配置

以下操作最好是在一个节点上进行：

[[email protected]~]# fdisk -cu /dev/sda

Device Boot Start End Blocks Id System

/dev/sda1 2048 3891199 1944576 83 Linux


[[email protected]~]# pvcreate /dev/sda1

[[email protected]~]# vgcreate -c y havg /dev/sda1 #-c y 表示让vg支持cluster

[[email protected]~]# vgdisplay

Clustered yes

[[email protected]~]# lvmconf --enable-cluster

[[email protected]~]# /etc/init.d/clvmd restart

#**lvm对与cluster的支持

#在创建vg后在另一个节点会同步vg

[[email protected]~]# lvcreate -L 1000M -n hadamo havg


如果出现如下问题：

Errorlocking on node 192.168.122.33: Volume group for uuid not found:vdfcn1i0cjjXRtw3ODjo54yopeUNtqAZZsy0Eh03PbVH9s5gFc98xNSqDLj0YFo4

Failed to activate new LV.

表示另一个节点上没有同步，可以在另一个节点上：

[[email protected]~]# iscsiadm --mode node --targetnameiqn.2013-09.desk82.example:server.target1 --portal192.168.122.82:3260 –logout

[[email protected]~]# iscsiadm -m node -l iqn.2013-09.desk82.example:server.target1

重新导入磁盘，确保磁盘同步，重新创建


创建完成后：

[[email protected]~]# lvs

LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert

hadamo havg -wi-a--- 1000.00m

[[email protected]~]# lvs

LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert

hadamo havg -wi-a--- 1000.00m


创建GFS文件系统：

mkfs.gfs2-p lock_dlm -t wangzi_1:gfs2 -j 3 /dev/havg/hadamo (格式化为gfs2)

-p定义为 DLM锁方式,如果不加此参数,当在两个系统中同时挂载此分区时就会像EXT3 格式一

样,两个系统的信息不能同步

-twangzi_1 DLM 锁所在的表名字,就是你的集群的名字gfs 分区的卷标

-j GFS 分区中最多支持多少个节点同时挂载,这个可以在使用中动态调整,通常设定值为节点数+1

wangzi_1:gfs2 ##wangzi_1为集群名，gfs2为一个标志，任意


[[email protected]]# mkfs.gfs2 -p lock_dlm -t wangzi_1:gfs2 -j 3/dev/havg/hadamo

Thiswill destroy any data on /dev/havg/hadamo.

Itappears to contain: symbolic link to `../dm-2'


Areyou sure you want to proceed? [y/n] y


Device: /dev/havg/hadamo

Blocksize: 4096

DeviceSize 0.98 GB (256000 blocks)

FilesystemSize: 0.98 GB (255997 blocks)

Journals: 3

ResourceGroups: 4

LockingProtocol: "lock_dlm"

LockTable: "wangzi_1:gfs2"

UUID: c62e7ef7-0179-8f8c-d6db-d15278ce4fc8


4.将GFS加入集群：


现将集群的服务组停掉：

RHCS+GFS(红帽HA+GFS)

添加资源：

RHCS+GFS(红帽HA+GFS)

在服务组apsche下添加资源：

RHCS+GFS(红帽HA+GFS)

启动服务组apsche：

RHCS+GFS(红帽HA+GFS)

[[email protected]]# /etc/init.d/httpd status

httpd(pid 30805) is running...

[[email protected]]# df

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/mapper/vol0-root

4031680 1150572 2676308 31% /

tmpfs 251304 32044 219260 13% /dev/shm

/dev/vda1 99150 28112 65918 30% /boot

/dev/mapper/vol0-home

516040 16796 473032 4% /home

/dev/mapper/havg-hadamo

1023824 397148 626676 39% /var/www/html


测试：

[[email protected]]# cd /var/www/html/

[[email protected]]# ls

[[email protected]]# echo "gfs.example.com" > index.html

RHCS+GFS(红帽HA+GFS)

关闭desk34的httpd服务：

[[email protected]]# df

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/mapper/vol0-root

4031680 1143516 2683364 30% /

tmpfs 251304 32044 219260 13% /dev/shm

/dev/vda1 99150 28096 65934 30% /boot

/dev/mapper/vol0-home

516040 16780 473048 4% /home

/dev/mapper/havg-hadamo

1023824 397152 626672 39% /var/www/html

切换到了desk33上；客户端访问不变：

RHCS+GFS(红帽HA+GFS)

5.GFS的在线扩展

[[email protected]~]# lvextend -L +500M /dev/havg/hadamo

Extending logical volume hadamo to 1.46 GiB

Logical volume hadamo successfully resized

[[email protected]~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vol0-root

3.9G 1.1G 2.6G 31% /

tmpfs 246M 32M 215M 13% /dev/shm

/dev/vda1 97M 28M 65M 30% /boot

/dev/mapper/vol0-home

504M 17M 462M 4% /home

/dev/mapper/havg-hadamo

1000M 388M 612M 39% /var/www/html

[[email protected]~]# gfs2_grow /dev/havg/hadamo

FS:Mount Point: /var/www/html

FS:Device: /dev/dm-2

FS:Size: 255997 (0x3e7fd)

FS:RG size: 63992 (0xf9f8)

DEV:Size: 384000 (0x5dc00)

Thefile system grew by 500MB.

gfs2_growcomplete.

[[email protected]~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vol0-root

3.9G 1.1G 2.6G 31% /

tmpfs 246M 32M 215M 13% /dev/shm

/dev/vda1 97M 28M 65M 30% /boot

/dev/mapper/vol0-home

504M 17M 462M 4% /home

/dev/mapper/havg-hadamo

1.5G 388M 1.1G 26% /var/www/html