一、环境说明
服务器IP地址
服务器版本
Redis版本
Redis-master
Redis-slave
10.10.18.10
centos7.5
5.0
6379
6380
10.10.18.11
centos7.5
5.0
6379
6380
10.10.18.12
centos7.5
5.0
6379
6380
搭建redis集群，建议至少准备3台服务器，共搭建6个节点，3个master,3个slave，并且要求3个master节点不能全部跑到同一台服务器上，保证节点安全，3台服务器的配置相同， 每个节点运行两个端口。第一列做主库，第二列做备库
每个Redis群集节点都需要打开两个TCP连接。用于为客户端提供服务的普通Redis TCP端口，例如6379，加上通过向数据端口添加10000获得的端口，因此示例中为16379。
默认数据存放结构图：
这个结构会随着机器的故障自动调整master的角色，有可能原来是slave的角色，会变成master。
三个slave可以都宕机，数据没有问题
Redis Cluster提供了一种运行Redis安装的方法，其中数据 在多个Redis节点之间自动分片。
Redis Cluster还在分区期间提供一定程度的可用性，实际上是在某些节点发生故障或无法通信时继续运行的能力。但是，如果发生较大的故障（例如，当大多数主设备不可用时），群集将停止运行。
二、系统环境配置
系统参数配置
修改最大可打开文件数
修改文件/etc/security/limits.conf
1* soft nofile 102400 2* hard nofile 102400
TCP监听队列大小
文件/etc/sysctl.conf中增加一行
net.core.somaxconn = 32767vm.overcommit_memory=1
然后执行命令“sysctl -p”以生效。
三、安装部署
下载redis5.0以上的版本
http://download.redis.io/releases/redis-5.0.3.tar.gz
1tar -xzvf redis-5.0.3.tar.gz –C /usr/local 2cd /usr/local/redis-5.0.3 3yum -y install gcc 4make MALLOC=libc 5mkdir –p /data/redis_data/ 6make install PREFIX=/data/redis_data/ 7mkdir –p /data/redis_data/6379/ 8 9cp /usr/local/redis-5.0.3/redis.conf /data/redis_data/6379/
修改：/data/redis_data/6379/redis.conf
1bind 10.10.18.10 2protected-mode no 3port 6379 4daemonize yes 5cluster-enabled yes 6cluster-config-file /data/redis_data/6379/node.conf #这个文件是自动生成的文件 7cluster-node-timeout 5000 8pidfile /data/redis_data/6379/redis.pid 9logfile "/data/redis_data/6379/redis.log" 10dir /data/redis_data/6379 11 12tcp-backlog 511 13timeout 0 14tcp-keepalive 300 15supervised no 16loglevel notice 17databases 16 18always-show-logo yes 19save 900 1 20save 300 10 21save 60 10000 22stop-writes-on-bgsave-error yes 23rdbcompression yes 24rdbchecksum yes 25dbfilename dump.rdb 26replica-serve-stale-data yes 27replica-read-only yes 28repl-diskless-sync no 29repl-diskless-sync-delay 5 30repl-disable-tcp-nodelay no 31replica-priority 100 32lazyfree-lazy-eviction no 33lazyfree-lazy-expire no 34lazyfree-lazy-server-del no 35replica-lazy-flush no 36appendonly yes 37appendfilename "appendonly.aof" 38appendfsync everysec 39no-appendfsync-on-rewrite no 40auto-aof-rewrite-percentage 100 41auto-aof-rewrite-min-size 64mb 42aof-load-truncated yes 43aof-use-rdb-preamble yes 44lua-time-limit 5000 45slowlog-log-slower-than 10000 46slowlog-max-len 128 47latency-monitor-threshold 0 48notify-keyspace-events "" 49hash-max-ziplist-entries 512 50hash-max-ziplist-value 64 51list-max-ziplist-size -2 52list-compress-depth 0 53set-max-intset-entries 512 54zset-max-ziplist-entries 128 55zset-max-ziplist-value 64 56hll-sparse-max-bytes 3000 57stream-node-max-bytes 4096 58stream-node-max-entries 100 59activerehashing yes 60client-output-buffer-limit normal 0 0 0 61client-output-buffer-limit replica 256mb 64mb 60 62client-output-buffer-limit pubsub 32mb 8mb 60 63hz 10 64dynamic-hz yes 65aof-rewrite-incremental-fsync yes 66rdb-save-incremental-fsync yes
修改：/data/redis_data/6379/redis.conf
1mkdir –p /data/redis_data/6380/ 2cp /data/redis_data/6379/redis.conf /data/redis_data/6380/ 3sed -i 's/6379/6380/g' /data/redis_data/6380/redis.conf
编写启动脚本 /data/redis_data/start-redis.sh
1#!/bin/sh 2REDIS_HOME=/data/redis_data 3$REDIS_HOME/bin/redis-server $REDIS_HOME/6379/redis.conf 4$REDIS_HOME/bin/redis-server $REDIS_HOME/6380/redis.conf 5 6chmod +x /data/redis_data/start-redis.sh
开启服务
查看各服务器的进程情况
1[root@redis01]# netstat -anpt |grep redis 2tcp 0 0 10.10.18.12:16379 0.0.0.0:* LISTEN 4328/redis-server 1 3tcp 0 0 10.10.18.12:16380 0.0.0.0:* LISTEN 4330/redis-server 1 4tcp 0 0 10.10.18.12:6379 0.0.0.0:* LISTEN 4328/redis-server 1 5tcp 0 0 10.10.18.12:6380 0.0.0.0:* LISTEN 4330/redis-server 1
创建启动集群脚本 /data/redis_data/redis-cluster.sh
1#!/bin/bash 2/data/redis_data/bin/redis-cli --cluster create 10.10.18.10:6379 10.10.18.11:6379 10.10.18.12:6379 10.10.18.10:6380 10.10.18.11:6380 10.10.18.12:6380 --cluster-replicas 1 3参数说明 4--cluster create：表示创建redis集群 5--cluster-replicas 1：表示为集群中的每一个主节点指定一个从节点，即一比一的复制。
三台服务器上都使用以上的安装步骤。
在任意一台服务器上执行，开启集群服务：
[root@redis01]# sh redis-cluster.sh
结果
1 1 >>> Performing hash slots allocation on 6 nodes... 2 2 Master[0] -> Slots 0 - 5460 3 3 Master[1] -> Slots 5461 - 10922 4 4 Master[2] -> Slots 10923 - 16383 5 5 Adding replica 10.10.18.11:6380 to 10.10.18.10:6379 6 6 Adding replica 10.10.18.10:6380 to 10.10.18.11:6379 7 7 Adding replica 10.10.18.12:6380 to 10.10.18.12:6379 8 8 >>> Trying to optimize slaves allocation for anti-affinity 9 9 [OK] Perfect anti-affinity obtained! 1010 M: c81b9adc68d5373237e9eaf3e5809bef42c5471a 10.10.18.10:6379 1111 slots:[0-5460] (5461 slots) master 1212 M: c4a495f621576910d934fa04da0896b39c0adf7a 10.10.18.11:6379 1313 slots:[5461-10922] (5462 slots) master 1414 M: baa5b1e2a769da1e37e3a4179ca362c9a26698c8 10.10.18.12:6379 1515 slots:[10923-16383] (5461 slots) master 1616 S: 8d315d8cc7f42afc5ac641003714f5e285822bff 10.10.18.10:6380 1717 replicates c4a495f621576910d934fa04da0896b39c0adf7a 1818 S: a7096a413407e77431c0d852c5106860d6d3184a 10.10.18.11:6380 1919 replicates baa5b1e2a769da1e37e3a4179ca362c9a26698c8 2020 S: 7f0c0858330bf107276d27c10f3e9c600aefa8ad 10.10.18.12:6380 2121 replicates c81b9adc68d5373237e9eaf3e5809bef42c5471a 2222 Can I set the above configuration? (type 'yes' to accept): yes 这里输入yes 2323 >>> Nodes configuration updated 2424 >>> Assign a different config epoch to each node 2525 >>> Sending CLUSTER MEET messages to join the cluster 2626 Waiting for the cluster to join 2727 .... 2828 >>> Performing Cluster Check (using node 10.10.18.10:6379) 2929 M: c81b9adc68d5373237e9eaf3e5809bef42c5471a 10.10.18.10:6379 3030 slots:[0-5460] (5461 slots) master 3131 1 additional replica(s) 3232 M: c4a495f621576910d934fa04da0896b39c0adf7a 10.10.18.11:6379 3333 slots:[5461-10922] (5462 slots) master 3434 1 additional replica(s) 3535 S: 8d315d8cc7f42afc5ac641003714f5e285822bff 10.10.18.10:6380 3636 slots: (0 slots) slave 3737 replicates c4a495f621576910d934fa04da0896b39c0adf7a 3838 S: 7f0c0858330bf107276d27c10f3e9c600aefa8ad 10.10.18.12:6380 3939 slots: (0 slots) slave 4040 replicates c81b9adc68d5373237e9eaf3e5809bef42c5471a 4141 S: a7096a413407e77431c0d852c5106860d6d3184a 10.10.18.11:6380 4242 slots: (0 slots) slave 4343 replicates baa5b1e2a769da1e37e3a4179ca362c9a26698c8 4444 M: baa5b1e2a769da1e37e3a4179ca362c9a26698c8 10.10.18.12:6379 4545 slots:[10923-16383] (5461 slots) master 4646 1 additional replica(s) 4747 [OK] All nodes agree about slots configuration. 4848 >>> Check for open slots... 4949 >>> Check slots coverage... 5050 [OK] All 16384 slots covered.
View Code
查看各服务器redis的状态
[root@redis01]# netstat -anpt | grep redis
结果：
1 1 tcp 0 0 10.10.18.12:16379 0.0.0.0:* LISTEN 4328/redis-server 1 2 2 tcp 0 0 10.10.18.12:16380 0.0.0.0:* LISTEN 4330/redis-server 1 3 3 tcp 0 0 10.10.18.12:6379 0.0.0.0:* LISTEN 4328/redis-server 1 4 4 tcp 0 0 10.10.18.12:6380 0.0.0.0:* LISTEN 4330/redis-server 1 5 5 tcp 0 0 10.10.18.12:46543 10.10.18.11:16380 ESTABLISHED 4330/redis-server 1 6 6 tcp 0 0 10.10.18.12:16379 10.10.18.10:40956 ESTABLISHED 4328/redis-server 1 7 7 tcp 0 0 10.10.18.12:16380 10.10.18.11:40559 ESTABLISHED 4330/redis-server 1 8 8 tcp 0 0 10.10.18.12:45872 10.10.18.10:16379 ESTABLISHED 4330/redis-server 1 9 9 tcp 0 0 10.10.18.12:45755 10.10.18.10:6379 ESTABLISHED 4330/redis-server 1 1010 tcp 0 0 10.10.18.12:16379 10.10.18.12:36457 ESTABLISHED 4328/redis-server 1 1111 tcp 0 0 10.10.18.12:39163 10.10.18.10:16379 ESTABLISHED 4328/redis-server 1 1212 tcp 0 0 10.10.18.12:43185 10.10.18.10:16380 ESTABLISHED 4328/redis-server 1 1313 tcp 0 0 10.10.18.12:42254 10.10.18.10:16380 ESTABLISHED 4330/redis-server 1 1414 tcp 0 0 10.10.18.12:35352 10.10.18.11:16379 ESTABLISHED 4330/redis-server 1 1515 tcp 0 0 10.10.18.12:41761 10.10.18.11:16379 ESTABLISHED 4328/redis-server 1 1616 tcp 0 0 10.10.18.12:36457 10.10.18.12:16379 ESTABLISHED 4330/redis-server 1 1717 tcp 0 0 10.10.18.12:16380 10.10.18.11:37557 ESTABLISHED 4330/redis-server 1 1818 tcp 0 0 10.10.18.12:6379 10.10.18.11:46722 ESTABLISHED 4328/redis-server 1 1919 tcp 0 0 10.10.18.12:16380 10.10.18.10:39318 ESTABLISHED 4330/redis-server 1 2020 tcp 0 0 10.10.18.12:16379 10.10.18.10:35113 ESTABLISHED 4328/redis-server 1 2121 tcp 0 0 10.10.18.12:16380 10.10.18.10:40872 ESTABLISHED 4330/redis-server 1 2222 tcp 0 0 10.10.18.12:16380 10.10.18.12:34377 ESTABLISHED 4330/redis-server 1 2323 tcp 0 0 10.10.18.12:16379 10.10.18.11:37717 ESTABLISHED 4328/redis-server 1 2424 tcp 0 0 10.10.18.12:43984 10.10.18.11:16380 ESTABLISHED 4328/redis-server 1 2525 tcp 0 0 10.10.18.12:16379 10.10.18.11:45994 ESTABLISHED 4328/redis-server 1 2626 tcp 0 0 10.10.18.12:34377 10.10.18.12:16380 ESTABLISHED 4328/redis-server 1
View Code
四、测试
设置redis密码
[root@redis01 bin]# /data/redis_data/bin/redis-cli -h 10.10.18.10 -p 6379 -c
结果：
110.10.18.10:6379> keys * 2(empty list or set) 310.10.18.10:6379> config set masterauth redis20190628 4OK 510.10.18.10:6379> config set requirepass redis20190628 6OK 710.10.18.10:6379> exit
查看所有的节点信息：
[root@redis01]# /data/redis_data/bin/redis-cli -h 10.10.18.10 -p 6380 -c -a 'redis20190628' cluster nodes
结果：
1Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 2c4a495f621576910d934fa04da0896b39c0adf7a 10.10.18.11:6379@16379 master - 0 1551081545000 2 connected 5461-10922 3c81b9adc68d5373237e9eaf3e5809bef42c5471a 10.10.18.10:6379@16379 master - 0 1551081544292 1 connected 0-5460 48d315d8cc7f42afc5ac641003714f5e285822bff 10.10.18.10:6380@16380 myself,slave c4a495f621576910d934fa04da0896b39c0adf7a 0 1551081543000 4 connected 5baa5b1e2a769da1e37e3a4179ca362c9a26698c8 10.10.18.12:6379@16379 master - 0 1551081543000 3 connected 10923-16383 67f0c0858330bf107276d27c10f3e9c600aefa8ad 10.10.18.12:6380@16380 slave c81b9adc68d5373237e9eaf3e5809bef42c5471a 0 1551081544292 6 connected 7a7096a413407e77431c0d852c5106860d6d3184a 10.10.18.11:6380@16380 slave baa5b1e2a769da1e37e3a4179ca362c9a26698c8 0 1551081543292 5 connected
添加测试数据
在任意一台机器上执行插入数据，然后在其它任意节点上都能获取到对应的值
1[root@redis01]# /data/redis_data/bin/redis-cli -h 10.10.18.10 -p 6380 -c -a 'redis20190628' 2Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 310.10.18.10:6380> set test10_6380 test 4[root@redis01]# /data/redis_data/bin/redis-cli -h 10.10.18.10 -p 6379 -c -a 'redis20190628' 5Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 610.10.18.10:6379> get test10_6380 7"test" 8 9[root@redis02]# /data/redis_data/bin/redis-cli -h 10.10.18.11 -p 6380 -c -a 'redis20190628' 10Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 1110.10.18.11:6380> get test10_6380 12-> Redirected to slot [2317] located at 10.10.18.10:6379 13"test"