PCS（Pacemaker/Corosync configuration system）命令配置群集⽰例：
⼀、建⽴群集：
1、配置群集节点的认证as the hacluster user: [shell]# pcs cluster auth node1 node2
2、创建⼀个⼆个节点的群集
[shell]# pcs cluster setup --name mycluster node1 node2 [shell]# pcs cluster start --all ## 启动群集
3、设置资源默认粘性（防⽌资源回切）
[shell]# pcs resource defaults resource-stickiness=100 [shell]# pcs resource defaults
4、设置资源超时时间
[shell]# pcs resource op defaults timeout=90s [shell]# pcs resource op defaults
5、⼆个节点时，忽略节点quorum功能
[shell]# pcs property set no-quorum-policy=ignore
6、没有 Fencing设备时，禁⽤STONITH 组件功能
在 stonith-enabled=\"false\" 的情况下，分布式锁管理器 (DLM) 等资源以及依赖DLM 的所有服务（例如 cLVM2、GFS2 和 OCFS2）都将⽆法启动。
[shell]# pcs property set stonith-enabled=false
[shell]# crm_verify -L -V ## 验证群集配置信息
⼆、建⽴群集资源
1、查看可⽤资源
[shell]# pcs resource list ## 查看⽀持资源列表，pcs resource list ocf:heartbeat
[shell]# pcs resource describe agent_name ## 查看资源使⽤参数，pcs resource describe ocf:heartbeat:IPaddr2
2、配置虚拟IP
[shell]# pcs resource create ClusterIP ocf:heartbeat:IPaddr2 \\
ip=\"192.168.10.15\" cidr_netmask=32 nic=eth0 op monitor interval=30s
3、配置Apache(httpd)
[shell]# pcs resource create WebServer ocf:heartbeat:apache \\ httpd=\"/usr/sbin/httpd\" configfile=\"/etc/httpd/conf/httpd.conf\" \\
statusurl=\"http://localhost/server-status\" op monitor interval=1min
4、配置Nginx
[shell]# pcs resource create WebServer ocf:heartbeat:nginx \\ httpd=\"/usr/sbin/nginx\" configfile=\"/etc/nginx/nginx.conf\" \\
statusurl=\"http://localhost/ngx_status\" op monitor interval=30s
5.1、配置FileSystem
[shell]# pcs resource create WebFS ocf:heartbeat:Filesystem \\ device=\"/dev/sdb1\" directory=\"/var/www/html\" fstype=\"ext4\"
[shell]# pcs resource create WebFS ocf:heartbeat:Filesystem \\ device=\"-U 32937d65eb\" directory=\"/var/www/html\" fstype=\"ext4\"
5.2、配置FileSystem-NFS
[shell]# pcs resource create WebFS ocf:heartbeat:Filesystem \\
device=\"192.168.10.18:/mysqldata\" directory=\"/var/lib/mysql\" fstype=\"nfs\" \\ options=\"-o username=your_name,password=your_password\" \\
op start timeout=60s op stop timeout=60s op monitor interval=20s timeout=60s
6、配置Iscsi
[shell]# pcs resource create WebData ocf:heartbeat:iscsi \\
portal=\"192.168.10.18\" target=\"iqn.2008-08.com.starwindsoftware:\" \\ op monitor depth=\"0\" timeout=\"30\" interval=\"120\"
[shell]# pcs resource create WebFS ocf:heartbeat:Filesystem \\
device=\"-U 32937d65eb\" directory=\"/var/www/html\" fstype=\"ext4\" options=\"_netdev\"
7、配置DRBD
[shell]# pcs resource create WebData ocf:linbit:drbd \\ drbd_resource=wwwdata op monitor interval=60s
[shell]# pcs resource master WebDataClone WebData \\
master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true
[shell]# pcs resource create WebFS ocf:heartbeat:Filesystem \\ device=\"/dev/drbd1\" directory=\"/var/www/html\" fstype=\"ext4\"
8、配置MySQL
[shell]# pcs resource create MySQL ocf:heartbeat:mysql \\
binary=\"/usr/bin/mysqld_safe\" config=\"/etc/my.cnf\" datadir=\"/var/lib/mysql\" \\ pid=\"/var/run/mysqld/mysql.pid\" socket=\"/tmp/mysql.sock\" \\
op start timeout=180s op stop timeout=180s op monitor interval=20s timeout=60s
9、配置Pingd，检测节点与⽬标的连接有效性
[shell]# pcs resource create PingCheck ocf:heartbeat:pingd \\ dampen=5s multiplier=100 host_list=\"192.168.10.1 router\" \\ op monitor interval=30s timeout=10s
10、创建资源clone，克隆的资源会在全部节点启动 [shell]# pcs resource clone PingCheck
[shell]# pcs resource clone ClusterIP clone-max=2 clone-node-max=2 globally-unique=true ## clone-max=2，数据包分成2路 [shell]# pcs resource update ClusterIP clusterip_hash=sourceip ## 指定响应请求的分配策略为：sourceip
三、调整群集资源
1、配置资源约束
[shell]# pcs resource group add WebSrvs ClusterIP ## 配置资源组，组中资源会在同⼀节点运⾏ [shell]# pcs resource group remove WebSrvs ClusterIP ## 移除组中的指定资源
[shell]# pcs resource master WebDataClone WebData ## 配置具有多个状态的资源，如 DRBD master/slave状态 [shell]# pcs constraint colocation add WebServer ClusterIP INFINITY ## 配置资源捆绑关系 [shell]# pcs constraint colocation remove WebServer ## 移除资源捆绑关系约束中资源 [shell]# pcs constraint order ClusterIP then WebServer ## 配置资源启动顺序
[shell]# pcs constraint order remove ClusterIP ## 移除资源启动顺序约束中资源 [shell]# pcs constraint ## 查看资源约束关系， pcs constraint --full
2、配置资源位置
[shell]# pcs constraint location WebServer prefers node11 ## 指定资源默认某个节点，node=50 指定增加的 score [shell]# pcs constraint location WebServer avoids node11 ## 指定资源避开某个节点，node=50 指定减少的 score [shell]# pcs constraint location remove location-WebServer ## 移除资源节点位置约束中资源ID，可⽤pcs config获取
[shell]# pcs constraint location WebServer prefers node11=INFINITY ## ⼿⼯移动资源节点，指定节点资源的 score of INFINITY [shell]# crm_simulate -sL ## 验证节点资源 score 值
3、修改资源配置
[shell]# pcs resource update WebFS ## 更新资源配置 [shell]# pcs resource delete WebFS ## 删除指定资源
4、管理群集资源
[shell]# pcs resource disable ClusterIP ## 禁⽤资源 [shell]# pcs resource enable ClusterIP ## 启⽤资源
[shell]# pcs resource failcount show ClusterIP ## 显⽰指定资源的错误计数 [shell]# pcs resource failcount reset ClusterIP ## 清除指定资源的错误计数 [shell]# pcs resource cleanup ClusterIP ## 清除指定资源的状态与错误计数
四、配置Fencing设备，启⽤STONITH
1、查询Fence设备资源
[shell]# pcs stonith list ## 查看⽀持Fence列表
[shell]# pcs stonith describe agent_name ## 查看Fence资源使⽤参数，pcs stonith describe fence_vmware_soap
2、配置fence设备资源
[shell]# pcs stonith create ipmi-fencing fence_ipmilan \\
pcmk_host_list=\"pcmk-1 pcmk-2\" ipaddr=\"10.0.0.1\" login=testuser passwd=acd123 \\ op monitor interval=60s
mark:
If the device does not support the standard port parameter or may provide additional ones, you may also need to set the specialpcmk_host_argument parameter. See man stonithd for details.
If the device does not know how to fence nodes based on their uname, you may also need to set the special pcmk_host_map parameter.See man stonithd for details.
If the device does not support the list command, you may also need to set the special pcmk_host_list and/or pcmk_host_check parameters.See man stonithd for details.
If the device does not expect the victim to be specified with the port parameter, you may also need to set the special pcmk_host_argumentparameter. See man stonithd for details.
example: pcmk_host_argument=\"uuid\" pcmk_host_map=\"node11:4;node12:5;node13:6\" pcmk_host_list=\"node11,node12\" pcmk_host_check=\"static-list\"
3、配置VMWARE (fence_vmware_soap)
特别说明：本次实例中使⽤了第3项（pcs stonith create vmware-fencing fence_vmware_soap）这个指定pcmk配置参数才能正常执⾏Fencing动作。
3.1、确认vmware虚拟机的状态：
[shell]# fence_vmware_soap -o list -a vcenter.example.com -l cluster-admin -p -z ## 获取虚拟机UUID [shell]# fence_vmware_soap -o status -a vcenter.example.com -l cluster-admin -p -z -U ## 查看状态 [shell]# fence_vmware_soap -o status -a vcenter.example.com -l cluster-admin -p -z -n
3.2、配置fence_vmware_soap
[shell]# pcs stonith create vmware-fencing-node11 fence_vmware_soap \\
action=\"reboot\" ipaddr=\"192.168.10.10\" login=\"vmuser\" passwd=\"vmuserpd\" ssl=\"1\" \\ port=\"node11\" shell_timeout=60s login_timeout=60s op monitor interval=90s
[shell]# pcs stonith create vmware-fencing-node11 fence_vmware_soap \\
action=\"reboot\" ipaddr=\"192.168.10.10\" login=\"vmuser\" passwd=\"vmuserpd\" ssl=\"1\" \\
uuid=\"421dec5f-c484-3d69-ddfb-65af46530581\" shell_timeout=60s login_timeout=60s op monitor interval=90s
[shell]# pcs stonith create vmware-fencing fence_vmware_soap \\
action=\"reboot\" ipaddr=\"192.168.10.10\" login=\"vmuser\" passwd=\"vmuserpd\" ssl=\"1\" \\
pcmk_host_argument=\"uuid\" pcmk_host_check=\"static-list\" pcmk_host_list=\"node11,node12\" \\
pcmk_host_map=\"node11:421dec5f-c484-3d69-ddfb-65af46530581;node12:421dec5f-c484-3d69-ddfb-65af46530582\" \\ shell_timeout=60s login_timeout=60s op monitor interval=90s
注：如果配置fence_vmware_soap设备时⽤port=vm name在测试时不能识别，则使⽤uuid=vm uuid代替；
建议使⽤ pcmk_host_argument、pcmk_host_map、pcmk_host_check、pcmk_host_list 参数指明节点与设备端⼝关系，格式：
pcmk_host_argument=\"uuid\" pcmk_host_map=\"node11:uuid4;node12:uuid5;node13:uuid6\" pcmk_host_list=\"node11,node12,node13\" pcmk_host_check=\"static-list\"
4、配置SCSI
[shell]# ls /dev/disk/by-id/wwn-* ## 获取Fencing磁盘UUID号，磁盘须未格式化 [shell]# pcs stonith create iscsi-fencing fence_scsi \\
action=\"reboot\" devices=\"/dev/disk/by-id/wwn-0x600e002\" meta provides=unfencing
5、配置DELL DRAC
[shell]# pcs stonith create dell-fencing-node11 fence_drac.....
6、管理 STONITH
[shell]# pcs resource clone vmware-fencing ## clone stonith资源，供多节点启动 [shell]# pcs property set stonith-enabled=true ## 启⽤ stonith 组件功能
[shell]# pcs stonith cleanup vmware-fencing ## 清除Fence资源的状态与错误计数 [shell]# pcs stonith fence node11 ## fencing指定节点
7、管理tomcat
[shell]#pcs resource create tomcat ersweb statusurl=http://127.0.0.1 java_home=/usr/java/jdk1.6.0_12/ catalina_home=/usr/local/Ers/tomcat/ op monitor interval=30s
五、群集操作命令
1、验证群集安装
[shell]# pacemakerd -F ## 查看pacemaker组件，ps axf | grep pacemaker [shell]# corosync-cfgtool -s ## 查看corosync序号
[shell]# corosync-cmapctl | grep members ## corosync 2.3.x [shell]# corosync-objctl | grep members ## corosync 1.4.x
2、查看群集资源
[shell]# pcs resource standards ## 查看⽀持资源类型 [shell]# pcs resource providers ## 查看资源提供商 [shell]# pcs resource agents ## 查看所有资源代理 [shell]# pcs resource list ## 查看⽀持资源列表 [shell]# pcs stonith list ## 查看⽀持Fence列表
[shell]# pcs property list --all ## 显⽰群集默认变量参数 [shell]# crm_simulate -sL ## 检验资源 score 值
3、使⽤群集脚本
[shell]# pcs cluster cib ra_cfg ## 将群集资源配置信息保存在指定⽂件
[shell]# pcs -f ra_cfg resource create ## 创建群集资源并保存在指定⽂件中（⽽⾮保存在运⾏配置） [shell]# pcs -f ra_cfg resource show ## 显⽰指定⽂件的配置信息，检查⽆误后 [shell]# pcs cluster cib-push ra_cfg ## 将指定配置⽂件加载到运⾏配置中
4、STONITH 设备操作
[shell]# stonith_admin -I ## 查询fence设备
[shell]# stonith_admin -M -a agent_name ## 查询fence设备的元数据，stonith_admin -M -a fence_vmware_soap [shell]# stonith_admin --reboot nodename ## 测试 STONITH 设备
5、查看群集配置
[shell]# crm_verify -L -V ## 检查配置有⽆错误 [shell]# pcs property ## 查看群集属性 [shell]# pcs stonith ## 查看stonith
[shell]# pcs constraint ## 查看资源约束 [shell]# pcs config ## 查看群集资源配置
[shell]# pcs cluster cib ## 以XML格式显⽰群集配置
6、管理群集
[shell]# pcs status ## 查看群集状态 [shell]# pcs status cluster [shell]# pcs status corosync
[shell]# pcs cluster stop [node11] ## 停⽌群集 [shell]# pcs cluster start --all ## 启动群集
[shell]# pcs cluster standby node11 ## 将节点置为后备standby状态，pcs cluster unstandby node11 [shell]# pcs cluster destroy [--all] ## 删除群集，[--all]同时恢复corosync.conf⽂件 [shell]# pcs resource cleanup ClusterIP ## 清除指定资源的状态与错误计数
[shell]# pcs stonith cleanup vmware-fencing ## 清除Fence资源的状态与错误计数
因篇幅问题不能全部显示，请点此查看更多更全内容