之前的文章已对RocketMQ做了详细介绍,这里就不再赘述了，下面是本人在测试和生产环境下RocketMQ3.4.6高可用集群的部署手册，在此分享下:
1) 基础环境
1ip地址 主机名 角色 2192.168.10.200 mq-master01 broker-a 3192.168.10.201 mq-master02 broker-b 4192.168.10.202 mq-slave01 broker-a-s 5192.168.10.239 mq-slave02 broker-b-s 6192.168.10.205 mq-console-nameserver nameserver、console 7 8生产环境准备5台虚拟机,如上,其中4台服务器为broker节点,2主2从;1台服务器作为nameserver和console节点. 9测试环境的话,可以只用两台,分别为broker的一主一从节点,nameserver和console服务一同部署到master节点上. 10 11关闭5台机器的iptables和selinux（所有节点机器上都要操作） 12[root@mq-master01 ~]# /etc/init.d/iptables stop 13[root@mq-master01 ~]# vim /etc/sysconfig/selinux 14...... 15SELINUX=disabled 16[root@mq-master01 ~]# setenforce 0 17[root@mq-master01 ~]# getenforce 18Permissive 19 205台机器都要安装统一的java环境 21将jdk-8u131-linux-x64.rpm下载到/usr/local/src目录下 22下载地址：https://pan.baidu.com/s/1pLaAjPp 23提取密码：x27s 24 25[root@mq-master01 ~]# java -version 26-bash: java: command not found 27 28如果自带了java环境,就卸载掉 29yum list installed |grep java 30yum -y remove java-1.7.0-openjdk* 31yum -y remove tzdata-java.noarch 32 33[root@mq-master01 ~]# cd /usr/local/src/ 34[root@mq-master01 src]# ll jdk-8u131-linux-x64_.rpm 35-rw-r--r-- 1 root root 169983496 Nov 19 2017 jdk-8u131-linux-x64_.rpm 36[root@mq-master01 src]# rpm -ivh jdk-8u131-linux-x64_.rpm --force 37 38[root@mq-master01 src]# vim /etc/profile 39...... 40JAVA_HOME=/usr/java/jdk1.8.0_131 41JAVA_BIN=/usr/java/jdk1.8.0_131/bin 42PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/bin:/sbin/ 43CLASSPATH=.:/lib/dt.jar:/lib/tools.jar 44export JAVA_HOME JAVA_BIN PATH CLASSPATH 45 46[root@mq-master01 src]# source /etc/profile 47[root@mq-master01 src]# java -version 48java version "1.8.0_131" 49Java(TM) SE Runtime Environment (build 1.8.0_131-b11) 50Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode) 51 525台机器最好前做hosts绑定，否则后续启动broker的时候，会报错诸如： 53java.net.UnknownHostException: mq-master01: mq-master01: Name or service not known 54 55[root@mq-master01 ~]# cat /etc/hosts 56...... 57192.168.10.200 mq-master01 58192.168.10.201 mq-master02 59192.168.10.202 mq-slave01 60192.168.10.239 mq-slave02 61192.168.10.205 mq-console-nameserver 62 635台机器的系统时间统一下 64[root@mq-master01 ~]# ntpdate ntp1.aliyun.com 65 6 Sep 12:59:51 ntpdate[1203]: adjust time server 120.25.115.20 offset -0.000831 sec 66 67本案例rocketmq3.4.6的相关软件包下载地址: https://pan.baidu.com/s/1VOYA60vIobtSxo9-9rTI6w 68提取密码: bxhr 69 70软件包分别是:rocketmq_admin.zip, rocketmq_namesrv.zip, rocketmq.zip 71下载后放到/usr/local/src目录下 72服务安装目录都在/data下
2) RocektMQ Broker双主双从环境部署(mq-master01,mq-master02,mq-slave01,mq-slave02四节点上操作)
11）安装rocketmq（四台broker节点机器操作一样） 2[root@mq-master01 ~]# cd /usr/local/src/ 3[root@mq-master01 src]# ll rocketmq.zip 4-rw-rw-r--. 1 1000 1000 16048060 May 22 10:33 rocketmq.zip 5[root@mq-master01 src]# unzip rocketmq.zip 6[root@mq-master01 src]# mv rocketmq /data/ 7 82) 配置rocektmq broker （四个节点机器都要操作）. 9[root@mq-master01 src]# cd /data/rocketmq/conf/2m-2s-sync/ 10[root@mq-master01 2m-2s-sync]# vim broker-a.properties 11brokerClusterName=Kevin-MQ-Cluster 12brokerName=broker-a 13brokerId=0 14namesrvAddr=192.168.10.205:20901 15autoCreateTopicEnable=false 16clusterTopicEnable=false 17brokerTopicEnable=false 18autoCreateSubscriptionGroup=false 19sendMessageThreadPoolNums=128 20pullMessageThreadPoolNums=128 21fetchNamesrvAddrByAddressServer=true 22filterServerNums=1 23serverWorkerThreads=32 24deleteWhen=04 25fileReservedTime=168 26brokerRole=SYNC_MASTER 27flushDiskType=SYNC_FLUSH 28#netty监听端口 29listenPort=20911 30 31==================================================================== 32mq-master02节点配置 33[root@mq-master02 src]# cat /data/rocketmq/conf/2m-2s-sync/broker-b.properties 34brokerClusterName=Kevin-MQ-Cluster 35brokerName=broker-b 36brokerId=0 37namesrvAddr=192.168.10.205:20901 38autoCreateTopicEnable=false 39clusterTopicEnable=false 40brokerTopicEnable=false 41autoCreateSubscriptionGroup=false 42sendMessageThreadPoolNums=128 43pullMessageThreadPoolNums=128 44fetchNamesrvAddrByAddressServer=true 45filterServerNums=1 46serverWorkerThreads=32 47deleteWhen=04 48fileReservedTime=168 49brokerRole=SYNC_MASTER 50flushDiskType=SYNC_FLUSH 51#netty监听端口 52listenPort=20911 53 54==================================================================== 55mq-slave01节点配置 56[root@mq-slave01 src]# cat /data/rocketmq/conf/2m-2s-sync/broker-a-s.properties 57brokerClusterName=Kevin-MQ-Cluster 58brokerName=broker-a 59brokerId=1 60namesrvAddr=192.168.10.205:20901 61autoCreateTopicEnable=false 62clusterTopicEnable=false 63brokerTopicEnable=false 64autoCreateSubscriptionGroup=false 65sendMessageThreadPoolNums=128 66pullMessageThreadPoolNums=128 67fetchNamesrvAddrByAddressServer=true 68filterServerNums=1 69serverWorkerThreads=32 70deleteWhen=04 71fileReservedTime=168 72brokerRole=SLAVE 73flushDiskType=SYNC_FLUSH 74#netty监听端口 75listenPort=20911 76 77==================================================================== 78mq-slave02节点配置 79[root@mq-slave02 src]# cat /data/rocketmq/conf/2m-2s-sync/broker-b-s.properties 80brokerClusterName=Kevin-MQ-Cluster 81brokerName=broker-b 82brokerId=1 83namesrvAddr=192.168.10.205:20901 84autoCreateTopicEnable=false 85clusterTopicEnable=false 86brokerTopicEnable=false 87autoCreateSubscriptionGroup=false 88sendMessageThreadPoolNums=128 89pullMessageThreadPoolNums=128 90fetchNamesrvAddrByAddressServer=true 91filterServerNums=1 92serverWorkerThreads=32 93deleteWhen=04 94fileReservedTime=168 95brokerRole=SLAVE 96flushDiskType=SYNC_FLUSH 97#netty监听端口 98listenPort=20911 99 1003) 启动NameServer （四台broker节点机器操作一样） 101只需要将/data/rocketmq/startBrokerAM.sh中的JAVA_HOME改成自己服务的java路径就行了. 102本案例的java家目录路径是/usr/java/jdk1.8.0_131 103 104[root@mq-master01 ~]# cat /data/rocketmq/startBrokerAM.sh |grep JAVA_HOME 105export JAVA_HOME=/data/software/jdk1.7.0_80 106[root@mq-master01 ~]# sed -i 's#/data/software/jdk1.7.0_80#/usr/java/jdk1.8.0_131#g' /data/rocketmq/startBrokerAM.sh 107[root@mq-master01 ~]# cat /data/rocketmq/startBrokerAM.sh |grep JAVA_HOME 108export JAVA_HOME=/usr/java/jdk1.8.0_131 109 110温馨提示： 111启动顺序：最好是先启动NameServer，再启动BrokerServer（启动BrokerServer之前关闭防火墙） 112停止顺序：最好先停止BrokerServer，再停止NameServer 113 114四台节点的启动脚本分别为: 115[root@mq-master01 rocketmq]# cat startBrokerAM.sh 116export JAVA_HOME=/usr/java/jdk1.8.0_131 117nohup sh ./alibaba-rocketmq/bin/mqbroker -c ./conf/2m-2s-sync/broker-a.properties > ./nohup.out 2>&1 & 118 119[root@mq-master02 rocketmq]# cat startBrokerAM.sh 120export JAVA_HOME=/usr/java/jdk1.8.0_131 121nohup sh ./alibaba-rocketmq/bin/mqbroker -c ./conf/2m-2s-sync/broker-b.properties > ./nohup.out 2>&1 & 122 123[root@mq-slave01 rocketmq]# cat startBrokerAM.sh 124export JAVA_HOME=/usr/java/jdk1.8.0_131 125nohup sh ./alibaba-rocketmq/bin/mqbroker -c ./conf/2m-2s-sync/broker-a-s.properties > ./nohup.out 2>&1 & 126 127[root@mq-slave02 rocketmq]# cat startBrokerAM.sh 128export JAVA_HOME=/usr/java/jdk1.8.0_131 129nohup sh ./alibaba-rocketmq/bin/mqbroker -c ./conf/2m-2s-sync/broker-b-s.properties > ./nohup.out 2>&1 & 130 131启动NameServer（四台broker节点机器操作一样） 132[root@mq-master01 ~]# cd /data/rocketmq/ #一定要先切换到该路径下 133[root@mq-master01 rocketmq]# /bin/bash /data/rocketmq/startBrokerAM.sh 134 135[root@mq-master01 rocketmq]# ps -ef|grep rocketmq 136root 16783 1 0 09:13 pts/1 00:00:00 sh ./alibaba-rocketmq/bin/mqbroker -c ./conf/2m-2s-sync/broker-a.properties 137root 16787 16783 0 09:13 pts/1 00:00:00 sh /data/rocketmq/alibaba-rocketmq/bin/runbroker.sh com.alibaba.rocketmq.broker.BrokerStartup -c ./conf/2m-2s-sync/broker-a.properties 138root 16790 16787 48 09:13 pts/1 00:00:01 /usr/java/jdk1.8.0_131/bin/java -Drocketmq.home.dir=/data/rocketmq/alibaba-rocketmq -Duser.home=/data/rocketmq -Xms2048M -Xmx2048M -XX:PermSize=512M -XX:MaxPermSize=512M -XX:NewSize=768M -XX:MaxNewSize=768M -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8 -XX:+DisableExplicitGC -verbose:gc -Xloggc:/data/rocketmq/logs/gclog/rmq_bk_gc.16787.log -XX:+PrintGCDetails -XX:-OmitStackTraceInFastThrow -Djava.ext.dirs=/data/rocketmq/alibaba-rocketmq/bin/../lib -cp .:/data/rocketmq/alibaba-rocketmq/bin/../conf: -Drocketmq.namesrv.domain=172.16.50.144 -Drocketmq.namesrv.domain.port=48080 -Drocketmq.namesrv.domain.subgroup=namesrv/getNSAddr.do com.alibaba.rocketmq.broker.BrokerStartup -c ./conf/2m-2s-sync/broker-a.properties 139root 16835 16088 0 09:13 pts/1 00:00:00 grep rocketmq 140 141[root@mq-master01 rocketmq]# lsof -i:20911 142COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME 143java 29934 root 79u IPv6 20211949 0t0 TCP *:20911 (LISTEN) 144java 29934 root 96u IPv6 20212542 0t0 TCP mq-master01:20911->mq-master02:53222 (ESTABLISHED) 145[root@mq-master01 rocketmq]# lsof -i:20901 146COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME 147java 29934 root 88u IPv6 20212051 0t0 TCP mq-master01:41510->mq-console-nameserver:20901 (ESTABLISHED) 148 149[root@mq-master01 rocketmq]# jps 15030235 Jps 15129934 BrokerStartup 152 153查看rockermq的版本: 154[root@mq-master01 rocketmq]# ll /data/rocketmq/alibaba-rocketmq/lib/ 155total 11836 156-rw-rw-r--. 1 root root 16088 Nov 28 2016 annotation-indexer-1.4.jar 157-rw-rw-r--. 1 root root 3159 Nov 28 2016 bridge-method-annotation-1.12.jar 158-rw-rw-r--. 1 root root 41123 Nov 28 2016 commons-cli-1.2.jar 159-rw-rw-r--. 1 root root 259600 Nov 28 2016 commons-codec-1.7.jar 160-rw-rw-r--. 1 root root 185140 Nov 28 2016 commons-io-2.4.jar 161-rw-rw-r--. 1 root root 284220 Nov 28 2016 commons-lang-2.6.jar 162-rw-rw-r--. 1 root root 2838580 Nov 28 2016 derby-10.10.2.0.jar 163-rw-rw-r--. 1 root root 409203 Nov 28 2016 fastjson-1.2.3.jar 164-rw-rw-r--. 1 root root 154710 Nov 28 2016 github-api-1.59.jar 165-rw-rw-r--. 1 root root 33483 Nov 28 2016 jackson-annotations-2.2.3.jar 166-rw-rw-r--. 1 root root 192699 Nov 28 2016 jackson-core-2.2.3.jar 167-rw-rw-r--. 1 root root 865838 Nov 28 2016 jackson-databind-2.2.3.jar 168-rw-rw-r--. 1 root root 544021 Nov 28 2016 javassist-3.7.ga.jar 169-rw-rw-r--. 1 root root 264600 Nov 28 2016 logback-classic-1.0.13.jar 170-rw-rw-r--. 1 root root 418870 Nov 28 2016 logback-core-1.0.13.jar 171-rw-rw-r--. 1 root root 1581066 Nov 28 2016 mockito-all-1.9.5.jar 172-rw-rw-r--. 1 root root 964879 Nov 28 2016 mysql-connector-java-5.1.31.jar 173-rw-rw-r--. 1 root root 1887979 Nov 28 2016 netty-all-4.0.25.Final.jar 174-rw-rw-r--. 1 root root 160098 Nov 28 2016 rocketmq-broker-3.4.6-lo-20161026.115524-467.jar 175-rw-rw-r--. 1 root root 241439 Nov 28 2016 rocketmq-client-3.4.6-lo-20160927.070341-319.jar 176-rw-rw-r--. 1 root root 213219 Nov 28 2016 rocketmq-common-3.4.6-lo-20160927.070334-319.jar 177-rw-rw-r--. 1 root root 38123 Nov 28 2016 rocketmq-filtersrv-3.4.6-lo-20160927.070417-318.jar 178-rw-rw-r--. 1 root root 33525 Nov 28 2016 rocketmq-namesrv-3.4.6-lo-20160927.070407-319.jar 179-rw-rw-r--. 1 root root 93612 Nov 28 2016 rocketmq-remoting-3.4.6-lo-20160927.070325-319.jar 180-rw-rw-r--. 1 root root 4088 Nov 28 2016 rocketmq-srvutil-3.4.6-lo-20160927.070351-319.jar 181-rw-rw-r--. 1 root root 157893 Nov 28 2016 rocketmq-store-3.4.6-lo-20160927.070348-319.jar 182-rw-rw-r--. 1 root root 147117 Nov 28 2016 rocketmq-tools-3.4.6-lo-20160927.070403-319.jar 183-rw-rw-r--. 1 root root 26084 Nov 28 2016 slf4j-api-1.7.5.jar
3) NameServer和Console部署(mq-console-nameserver节点上操作)
1安装配置NameServer 2[root@mq-console-nameserver src]# cd 3[root@mq-console-nameserver ~]# cd /usr/local/src/ 4[root@mq-console-nameserver src]# ll rocketmq_namesrv.zip 5-rw-rw-r-- 1 1000 1000 11802959 May 22 11:03 rocketmq_namesrv.zip 6[root@mq-console-nameserver src]# unzip rocketmq_namesrv.zip 7[root@mq-console-nameserver src]# mv rocketmq_namesrv /data/ 8[root@mq-console-nameserver src]# vim /data/rocketmq_namesrv/conf/namesrv.properties 9listenPort=20901 10 11修改JAVA_HOME 12[root@mq-console-nameserver src]# sed -i 's#/data/software/jdk1.7.0_80#/usr/java/jdk1.8.0_131#g' /data/rocketmq_namesrv/startNameServer.sh 13[root@mq-console-nameserver src]# cat /data/rocketmq_namesrv/startNameServer.sh 14export JAVA_HOME=/usr/java/jdk1.8.0_131 15nohup sh ./alibaba-rocketmq/bin/mqnamesrv -c ./conf/namesrv.properties > ./nohup.out 2>&1 & 16 17启动NameServer 18[root@mq-console-nameserver src]# cd /data/rocketmq_namesrv/ 19[root@mq-console-nameserver rocketmq_namesrv]# sh startNameServer.sh 20 21[root@mq-console-nameserver rocketmq_namesrv]# ps -ef|grep mqnamesrv 22root 17455 1 0 09:23 pts/1 00:00:00 sh ./alibaba-rocketmq/bin/mqnamesrv -c ./conf/namesrv.properties 23root 17555 24306 0 09:23 pts/1 00:00:00 grep mqnamesrv 24[root@mq-console-nameserver rocketmq_namesrv]# lsof -i:20901 25COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME 26java 17461 root 47u IPv6 40870224 0t0 TCP *:20901 (LISTEN) 27 28 29================================================== 30安装部署Console监控平台 31[root@mq-console-nameserver ~]# cd /usr/local/src/ 32[root@mq-console-nameserver src]# ll rocketmq_admin.zip 33-rw-rw-r-- 1 1000 1000 41574127 May 22 11:03 rocketmq_admin.zip 34[root@mq-console-nameserver src]# unzip rocketmq_admin.zip 35 36服务端口(如下改成48080) 37[root@mq-console-nameserver src]# vim /data/rocketmq_admin/apache-tomcat-8.0.32-rocketmq-console/conf/server.xml 38...... 39 <Connector port="48080" protocol="HTTP/1.1" 40 connectionTimeout="20000" 41 redirectPort="28443" /> 42 43[root@mq-console-nameserver src]# mv rocketmq_admin /data/ 44[root@mq-console-nameserver src]# cd /data/rocketmq_admin/ 45[root@mq-console-nameserver rocketmq_admin]# cd apache-tomcat-8.0.32-rocketmq-console/webapps/rocketmq/WEB-INF/classes/ 46[root@mq-console-nameserver classes]# vim config.properties 47rocketmq.namesrv.addr=192.168.10.205:20901 48throwDone=true 49 50启动console服务 51[root@mq-console-nameserver classes]# /data/rocketmq_admin/apache-tomcat-8.0.32-rocketmq-console/bin/startup.sh 52[root@mq-console-nameserver classes]# lsof -i:48080 53COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME 54java 19338 root 46u IPv6 40883769 0t0 TCP *:48080 (LISTEN)
访问RockerMQ Console监控平台http://192.168.10.205:48080/rocketmq/cluster/list.do,如下截图:
