[TOC]
我叫张贺，贪财好色。一名合格的LINUX运维工程师，专注于LINUX的学习和研究，曾负责某中型企业的网站运维工作，爱好佛学和跑步。 个人博客：传送阵 笔者微信：
zhanghe15069028807，非诚勿扰。
Redis集群
理论
Redis集群起码是三个服务器做，每个服务器里面都有两个实例，两个实例并不能是单纯的一主一从，而是错开，防止一个服务器挂了之后，数据丢失，如下图所示：
当集群安装好之后，会生成16384个槽位用于存储键值，其中0--5500在master1这个槽位上，5501--11000在master2槽位上，11001--16384在第三个槽位上，如下所示：
当我们存储一个键值的时候，redis会通过一个公式计算出一个值，假如这个值 是6666的话，那么这个键值就存储到master2这个实例上，依次类推，存的时候是这样存，取的时候同样也是这样取。
客户端在连接的时候无论连接哪一个节点都可以，如果本节点上没有要存取的键值的话，本节点会帮客户端转发，客户端不需要重新连接。
万一其中的一个主挂了，并不是从立马主顶上，而是剩下的两个主一同投票来决定，并不是利用了“哨兵sentinel”的机制。
安装
我们操作的时候用一台服务器，然后开6个实例：7000-----7005
1[root@NFS ~]# yum -y install ruby rubygems 2 3//默认源在国外 4[root@NFS ~]# gem sources -l 5*** CURRENT SOURCES *** 6https://rubygems.org/ 7 8//换成阿里云的 9[root@NFS ~]# gem sources -a https://mirrors.aliyun.com/rubygems/ 10https://mirrors.aliyun.com/rubygems/ added to sources 11 12//移除原来的 13[root@NFS ~]# gem sources --remove https://rubygems.org/ 14https://rubygems.org/ removed from sources 15 16//安装集群插件 17[root@NFS ~]# gem install redis -v 3.3.3 18Fetching: redis-3.3.3.gem (100%) 19Successfully installed redis-3.3.3 20Parsing documentation for redis-3.3.3 21Installing ri documentation for redis-3.3.3 221 gem installed 23 24//只有一个源了 25[root@NFS ~]# gem sources -l 26*** CURRENT SOURCES *** 27 28https://mirrors.aliyun.com/rubygems/ 29 30 31//节点准备 32mkdir /nosql/700{0..5} 33 34vim /nosql/7000/redis.conf 35port 7000 36daemonize yes 37pidfile /nosql/7000/redis.pid 38loglevel notice 39logfile "/nosql/7000/redis.log" 40dbfilename dump.rdb 41dir /nosql/7000 42protected-mode no 43cluster-enabled yes 44cluster-config-file nodes.conf 45cluster-node-timeout 5000 46appendonly yes 47 48vim /nosql/7001/redis.conf 49port 7001 50daemonize yes 51pidfile /nosql/7001/redis.pid 52loglevel notice 53logfile "/nosql/7001/redis.log" 54dbfilename dump.rdb 55dir /nosql/7001 56protected-mode no 57cluster-enabled yes 58cluster-config-file nodes.conf 59cluster-node-timeout 5000 60appendonly yes 61 62vim /nosql/7002/redis.conf 63port 7002 64daemonize yes 65pidfile /nosql/7002/redis.pid 66loglevel notice 67logfile "/nosql/7002/redis.log" 68dbfilename dump.rdb 69dir /nosql/7002 70protected-mode no 71cluster-enabled yes 72cluster-config-file nodes.conf 73cluster-node-timeout 5000 74appendonly yes 75 76vim /nosql/7003/redis.conf 77port 7003 78daemonize yes 79pidfile /nosql/7003/redis.pid 80loglevel notice 81logfile "/nosql/7003/redis.log" 82dbfilename dump.rdb 83dir /nosql/7003 84protected-mode no 85cluster-enabled yes 86cluster-config-file nodes.conf 87cluster-node-timeout 5000 88appendonly yes 89 90vim /nosql/7004/redis.conf 91port 7004 92daemonize yes 93pidfile /nosql/7004/redis.pid 94loglevel notice 95logfile "/nosql/7004/redis.log" 96dbfilename dump.rdb 97dir /nosql/7004 98protected-mode no 99cluster-enabled yes 100cluster-config-file nodes.conf 101cluster-node-timeout 5000 102appendonly yes 103 104vim /nosql/7005/redis.conf 105port 7005 106daemonize yes 107pidfile /nosql/7005/redis.pid 108loglevel notice 109logfile "/nosql/7005/redis.log" 110dbfilename dump.rdb 111dir /nosql/7005 112protected-mode no 113cluster-enabled yes 114cluster-config-file nodes.conf 115cluster-node-timeout 5000 116appendonly yes 117 118redis-server /nosql/7000/redis.conf 119redis-server /nosql/7001/redis.conf 120redis-server /nosql/7002/redis.conf 121redis-server /nosql/7003/redis.conf 122redis-server /nosql/7004/redis.conf 123redis-server /nosql/7005/redis.conf 124 125[root@NFS ~]# ps -ef | grep redis 126root 28437 1 0 14:07 ? 00:00:00 redis-server *:7000 [cluster] 127root 28450 1 0 14:09 ? 00:00:00 redis-server *:7001 [cluster] 128root 28452 1 0 14:09 ? 00:00:00 redis-server *:7002 [cluster] 129root 28456 1 0 14:09 ? 00:00:00 redis-server *:7003 [cluster] 130root 28458 1 0 14:09 ? 00:00:00 redis-server *:7004 [cluster] 131root 28466 1 0 14:09 ? 00:00:00 redis-server *:7005 [cluster] 132 133//将节点加入集群管理,--replicas 1代表一个主带一个从，前三个是主，后三个是对应的是从 134redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 135 136>>> Creating cluster 137>>> Performing hash slots allocation on 6 nodes... 138Using 3 masters: 139127.0.0.1:7000 140127.0.0.1:7001 141127.0.0.1:7002 142Adding replica 127.0.0.1:7003 to 127.0.0.1:7000 143Adding replica 127.0.0.1:7004 to 127.0.0.1:7001 144Adding replica 127.0.0.1:7005 to 127.0.0.1:7002 145M: 7517120d8c2b37de8f94cfeb614d8032e24fc129 127.0.0.1:7000 146 slots:0-5460 (5461 slots) master 147M: 367370edab7d5aef62fff5447f0f35e6ddbc8728 127.0.0.1:7001 148 slots:5461-10922 (5462 slots) master 149M: cab39e65a84cbbb8db8ea6d6c04f998bba62876c 127.0.0.1:7002 150 slots:10923-16383 (5461 slots) master 151S: 66af6f975ff30f4fa86dad5250fc2cbaa8dab592 127.0.0.1:7003 152 replicates 7517120d8c2b37de8f94cfeb614d8032e24fc129 153S: bdc7a5dfc50b1d96f5510a5ee01b5ec85411c39a 127.0.0.1:7004 154 replicates 367370edab7d5aef62fff5447f0f35e6ddbc8728 155S: 1a26aa7767c4db8ebe0cf0d937e6afc44eba531e 127.0.0.1:7005 156 replicates cab39e65a84cbbb8db8ea6d6c04f998bba62876c 157Can I set the above configuration? (type 'yes' to accept): yes 158>>> Nodes configuration updated 159>>> Assign a different config epoch to each node 160>>> Sending CLUSTER MEET messages to join the cluster 161Waiting for the cluster to join... 162>>> Performing Cluster Check (using node 127.0.0.1:7000) 163M: 7517120d8c2b37de8f94cfeb614d8032e24fc129 127.0.0.1:7000 164 slots:0-5460 (5461 slots) master 165 1 additional replica(s) 166S: bdc7a5dfc50b1d96f5510a5ee01b5ec85411c39a 127.0.0.1:7004 167 slots: (0 slots) slave 168 replicates 367370edab7d5aef62fff5447f0f35e6ddbc8728 169S: 66af6f975ff30f4fa86dad5250fc2cbaa8dab592 127.0.0.1:7003 170 slots: (0 slots) slave 171 replicates 7517120d8c2b37de8f94cfeb614d8032e24fc129 172M: 367370edab7d5aef62fff5447f0f35e6ddbc8728 127.0.0.1:7001 173 slots:5461-10922 (5462 slots) master 174 1 additional replica(s) 175M: cab39e65a84cbbb8db8ea6d6c04f998bba62876c 127.0.0.1:7002 176 slots:10923-16383 (5461 slots) master 177 1 additional replica(s) 178S: 1a26aa7767c4db8ebe0cf0d937e6afc44eba531e 127.0.0.1:7005 179 slots: (0 slots) slave 180 replicates cab39e65a84cbbb8db8ea6d6c04f998bba62876c 181[OK] All nodes agree about slots configuration. 182>>> Check for open slots... 183>>> Check slots coverage... 184[OK] All 16384 slots covered. 185 186上面的内容有对应关系： 1877000----7003 1887001----7004 1897002----7005 190 191注意还有槽位 192 193//查看集群主的状态 194[root@NFS ~]# redis-cli -p 7000 cluster nodes | grep master 195367370edab7d5aef62fff5447f0f35e6ddbc8728 127.0.0.1:7001 master - 0 1578378191503 2 connected 5461-10922 196cab39e65a84cbbb8db8ea6d6c04f998bba62876c 127.0.0.1:7002 master - 0 1578378191503 3 connected 10923-16383 1977517120d8c2b37de8f94cfeb614d8032e24fc129 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460 198 199//查看集群从的状态 200[root@NFS ~]# redis-cli -p 7000 cluster nodes | grep slave 201bdc7a5dfc50b1d96f5510a5ee01b5ec85411c39a 127.0.0.1:7004 slave 367370edab7d5aef62fff5447f0f35e6ddbc8728 0 1578378287351 5 connected 20266af6f975ff30f4fa86dad5250fc2cbaa8dab592 127.0.0.1:7003 slave 7517120d8c2b37de8f94cfeb614d8032e24fc129 0 1578378287855 4 connected 2031a26aa7767c4db8ebe0cf0d937e6afc44eba531e 127.0.0.1:7005 slave cab39e65a84cbbb8db8ea6d6c04f998bba62876c 0 1578378286845 6 connected
管理
增加新-主节点
1mkdir /nosql/7006 2mkdir /nosql/7007 3 4vim /nosql/7006/redis.conf 5port 7006 6daemonize yes 7pidfile /nosql/7006/redis.pid 8loglevel notice 9logfile "/nosql/7006/redis.log" 10dbfilename dump.rdb 11dir /nosql/7006 12protected-mode no 13cluster-enabled yes 14cluster-config-file nodes.conf 15cluster-node-timeout 5000 16appendonly yes 17 18vim /nosql/7007/redis.conf 19port 7007 20daemonize yes 21pidfile /nosql/7007/redis.pid 22loglevel notice 23logfile "/nosql/7007/redis.log" 24dbfilename dump.rdb 25dir /nosql/7007 26protected-mode no 27cluster-enabled yes 28cluster-config-file nodes.conf 29cluster-node-timeout 5000 30appendonly yes 31 32redis-server /nosql/7007/redis.conf 33redis-server /nosql/7006/redis.conf 34 35//把7006加入7000所在的集群 36redis-trib.rb add-node 127.0.0.1:7006 127.0.0.1:7000 37[ERR] Node 127.0.0.1:7006 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0. 38解决办法：进入新加入的节点，flushdb，然后shutdown，然后重启，再加一遍就会成功。 39//查看一下，7006没有槽位信息 40[root@NFS ~]# redis-cli -p 7000 cluster nodes | grep master 41367370edab7d5aef62fff5447f0f35e6ddbc8728 127.0.0.1:7001 master - 0 1578382465839 2 connected 5461-10922 42cab39e65a84cbbb8db8ea6d6c04f998bba62876c 127.0.0.1:7002 master - 0 1578382465333 3 connected 10923-16383 437517120d8c2b37de8f94cfeb614d8032e24fc129 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460 442860170269d03fc5c543eb33cbff0e2c54b26266 127.0.0.1:7006 master - 0 1578382464329 0 connected 45 46 47//转移一部分槽位（重新分片）给7006，让共平均分配，应该把运行中的应用在停止一下，计算一下用总的除以4等于4096 48redis-trib.rb reshard 127.0.0.1:7000 49How many slots do you want to move (from 1 to 16384)? 4096 #给新的节点多少槽位 50What is the receiving node ID? 2860170269d03fc5c543eb33cbff0e2c54b26266 #新节点的ID 51Please enter all the source node IDs. 52 Type 'all' to use all the nodes as source nodes for the hash slots. 53 Type 'done' once you entered all the source nodes IDs. 54Source node #1:all #源是所有集群内节点 55Do you want to proceed with the proposed reshard plan (yes/no)? yes 56 57[root@NFS ~]# redis-cli -p 7000 cluster nodes | grep master #每人给了一点 58367370edab7d5aef62fff5447f0f35e6ddbc8728 127.0.0.1:7001 master - 0 1578383170298 2 connected 6827-10922 59cab39e65a84cbbb8db8ea6d6c04f998bba62876c 127.0.0.1:7002 master - 0 1578383172313 3 connected 12288-16383 607517120d8c2b37de8f94cfeb614d8032e24fc129 127.0.0.1:7000 myself,master - 0 0 1 connected 1365-5460 612860170269d03fc5c543eb33cbff0e2c54b26266 127.0.0.1:7006 master - 0 1578383171810 7 connected 0-1364 5461-6826 10923-12287
### 增加新-从节点
1[root@NFS ~]# redis-trib.rb add-node --slave --master-id 2860170269d03fc5c543eb33cbff0e2c54b26266 127.0.0.1:7007 127.0.0.1:7000 2[ERR] Node 127.0.0.1:7007 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0. 3把７００７关闭，然后除了配置文件之外全部清除，最后重启，最后再加一次就成功了。 4 5[root@NFS 7007]# redis-cli -p 7000 cluster nodes | grep slave 6abdacebbecf2c4c948dec63601ff3f8eb6c5a64d 127.0.0.1:7007 slave 2860170269d03fc5c543eb33cbff0e2c54b26266 0 1578384755252 7 connected 7bdc7a5dfc50b1d96f5510a5ee01b5ec85411c39a 127.0.0.1:7004 slave 367370edab7d5aef62fff5447f0f35e6ddbc8728 0 1578384755756 5 connected 866af6f975ff30f4fa86dad5250fc2cbaa8dab592 127.0.0.1:7003 slave 7517120d8c2b37de8f94cfeb614d8032e24fc129 0 1578384755756 4 connected 91a26aa7767c4db8ebe0cf0d937e6afc44eba531e 127.0.0.1:7005 slave cab39e65a84cbbb8db8ea6d6c04f998bba62876c 0 1578384755252 6 connected 10
删除节点
删除节点7006之前，要把槽位移动走，不然把槽位也会一同给删除了，又要重新分片。
1redis-trib.rb reshard 127.0.0.1:7000 2# 移动多少，当时分配了4096个，现在也就要移动走这么多 3# 让你选择接收的节点，填7000的ID 4# 让你选择源节点，也就是要删除哪个节点，填7006的节点 5# done 6# yes 7 8[root@NFS ~]# redis-cli -p 7000 cluster nodes | grep master 9367370edab7d5aef62fff5447f0f35e6ddbc8728 127.0.0.1:7001 master - 0 1578385715675 2 connected 6827-10922 10cab39e65a84cbbb8db8ea6d6c04f998bba62876c 127.0.0.1:7002 master - 0 1578385716180 3 connected 12288-16383 117517120d8c2b37de8f94cfeb614d8032e24fc129 127.0.0.1:7000 myself,master - 0 0 8 connected 0-6826 10923-12287 122860170269d03fc5c543eb33cbff0e2c54b26266 127.0.0.1:7006 master - 0 1578385716683 7 connected 13 14//删除主节点 15[root@NFS ~]# redis-trib.rb del-node 127.0.0.1:7006 2860170269d03fc5c543eb33cbff0e2c54b26266 16 17[root@NFS ~]# redis-cli -p 7000 cluster nodes | grep slave 18abdacebbecf2c4c948dec63601ff3f8eb6c5a64d 127.0.0.1:7007 slave 7517120d8c2b37de8f94cfeb614d8032e24fc129 0 1578385840781 8 connected 19bdc7a5dfc50b1d96f5510a5ee01b5ec85411c39a 127.0.0.1:7004 slave 367370edab7d5aef62fff5447f0f35e6ddbc8728 0 1578385841286 5 connected 2066af6f975ff30f4fa86dad5250fc2cbaa8dab592 127.0.0.1:7003 slave 7517120d8c2b37de8f94cfeb614d8032e24fc129 0 1578385841286 8 connected 211a26aa7767c4db8ebe0cf0d937e6afc44eba531e 127.0.0.1:7005 slave cab39e65a84cbbb8db8ea6d6c04f998bba62876c 0 1578385841790 6 connected 22 23//删除从节点 24[root@NFS ~]# redis-trib.rb del-node 127.0.0.1:7007 abdacebbecf2c4c948dec63601ff3f8eb6c5a64d