MongoDB分片搭建

· 2015年度PG大象会报名地址：http://postgres2015.eventdove.com/

· PostgreSQL中国社区： http://postgres.cn/index.php/home

· PostgreSQL专业1群： 3336901（已满）

· PostgreSQL专业2群： 100910388

· PostgreSQL专业3群： 150657323

· PostgresChina微信公众号     PostgreSQL用户会微博二维码

一、环境

1$ cat /etc/redhat-release 
2CentOS Linux release 7.0.1406 (Core) 
3$ uname -a
4Linux zhaopin-2-201 3.10.0-123.el7.x86_64 #1 SMP Mon Jun 30 12:09:22 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
5$ mongo --version
6MongoDB shell version: 3.0.6
7
8node1: 172.30.2.201
9node2: 172.30.2.202
10node3: 172.30.2.203

二、配置Shard Server

在3个节点分别执行:

1. 创建目录

$ sudo mkdir -p /data/mongodb/{data/{sh0,sh1},backup/{sh0,sh1},log/{sh0,sh1},conf/{sh0,sh1}}

2.准备配置文件

第一个分片：

1$ sudo vim /data/mongodb/conf/sh0/mongodb.conf
2# base
3port = 27010
4maxConns = 800 
5filePermissions = 0700
6fork = true
7noauth = true
8directoryperdb = true
9dbpath = /data/mongodb/data/sh0
10pidfilepath = /data/mongodb/data/sh0/mongodb.pid
11oplogSize = 10
12journal = true
13# security
14nohttpinterface = true
15rest = false
16# log 
17logpath = /data/mongodb/log/sh0/mongodb.log
18logRotate = rename
19logappend = true
20slowms = 50
21replSet = sh0 
22shardsvr = true

第二个分片：

1$ sudo vim /data/mongodb/conf/sh1/mongodb.conf
2# base
3port = 27011
4maxConns = 800 
5filePermissions = 0700
6fork = true
7noauth = true
8directoryperdb = true
9dbpath = /data/mongodb/data/sh1
10pidfilepath = /data/mongodb/data/sh1/mongodb.pid
11oplogSize = 10
12journal = true
13# security
14nohttpinterface = true
15rest = false
16# log
17logpath = /data/mongodb/log/sh1/mongodb.log
18logRotate = rename
19logappend = true
20slowms = 50
21replSet = sh1
22shardsvr = true

3.启动Shard Server

1$ sudo /opt/mongodb/bin/mongod --config /data/mongodb/conf/sh0/mongodb.conf
2about to fork child process, waiting until server is ready for connections.
3forked process: 41492
4child process started successfully, parent exiting
5$ sudo /opt/mongodb/bin/mongod --config /data/mongodb/conf/sh1/mongodb.conf
6about to fork child process, waiting until server is ready for connections.
7forked process: 41509
8child process started successfully, parent exiting
9$ ps aux | grep mongo | grep -v grep
10root     41492  0.5  0.0 518016 54604 ?        Sl   10:09   0:00 /opt/mongodb/bin/mongod --config /data/mongodb/conf/sh0/mongodb.conf
11root     41509  0.5  0.0 516988 51824 ?        Sl   10:09   0:00 /opt/mongodb/bin/mongod --config /data/mongodb/conf/sh1/mongodb.conf
12$ mongo --port 27010
13MongoDB shell version: 3.0.6
14connecting to: 127.0.0.1:27010/test
15>
16bye
17$ mongo --port 27011
18MongoDB shell version: 3.0.6
19connecting to: 127.0.0.1:27011/test
20>
21bye

三、配置Config Server

在3个节点分别执行:

1.创建目录

$ sudo mkdir -p /data/mongodb/{data/cf0,backup/cf0,log/cf0,conf/cf0}

2.准备配置文件

1$ sudo vim /data/mongodb/conf/cf0/config.conf
2# base
3port = 27000
4maxConns = 800
5filePermissions = 0700
6fork = true
7noauth = true
8directoryperdb = true
9dbpath = /data/mongodb/data/cf0
10pidfilepath = /data/mongodb/data/cf0/config.pid
11oplogSize = 10
12journal = true
13# security
14nohttpinterface = true
15rest = false
16# log
17logpath = /data/mongodb/log/cf0/config.log
18logRotate = rename
19logappend = true
20slowms = 50
21configsvr = true

3.启动

1$ sudo /opt/mongodb/bin/mongod --config /data/mongodb/conf/cf0/config.conf
2about to fork child process, waiting until server is ready for connections.
3forked process: 41759
4child process started successfully, parent exiting
5$ ps aux | grep mongo | grep -v grep
6root     41492  0.3  0.0 518016 54728 ?        Sl   10:09   0:06 /opt/mongodb/bin/mongod --config /data/mongodb/conf/sh0/mongodb.conf
7root     41509  0.3  0.0 518016 54760 ?        Sl   10:09   0:06 /opt/mongodb/bin/mongod --config /data/mongodb/conf/sh1/mongodb.conf
8root     41855  0.4  0.0 467828 51684 ?        Sl   10:25   0:03 /opt/mongodb/bin/mongod --config /data/mongodb/conf/cf0/config.conf

四、配置Query Routers

在3个节点分别执行:

1.创建目录

$ sudo mkdir -p /data/mongodb/{data/ms0,backup/ms0,log/ms0,conf/ms0}

2.准备配置文件

1$ sudo vim /data/mongodb/conf/ms0/mongos.conf
2# base
3port = 30000
4maxConns = 800
5filePermissions = 0700
6fork = true
7pidfilepath = /data/mongodb/data/ms0/mongos.pid
8# log
9logpath = /data/mongodb/log/ms0/mongos.log
10logRotate = rename
11logappend = true
12configdb = 172.30.2.201:27000,172.30.2.202:27000,172.30.2.203:27000

3.启动

1$ sudo /opt/mongodb/bin/mongos --config /data/mongodb/conf/ms0/mongos.conf
2about to fork child process, waiting until server is ready for connections.
3forked process: 42233
4child process started successfully, parent exiting
5$ ps aux | grep mongo | grep -v grep
6root     41492  0.3  0.0 518016 54728 ?        Sl   10:09   0:06 /opt/mongodb/bin/mongod --config /data/mongodb/conf/sh0/mongodb.conf
7root     41509  0.3  0.0 518016 54760 ?        Sl   10:09   0:07 /opt/mongodb/bin/mongod --config /data/mongodb/conf/sh1/mongodb.conf
8root     41855  0.4  0.0 546724 37812 ?        Sl   10:25   0:03 /opt/mongodb/bin/mongod --config /data/mongodb/conf/cf0/config.conf
9root     41870  0.4  0.0 546724 38188 ?        Sl   10:25   0:03 /opt/mongodb/bin/mongod --conf
10root     42233  0.5  0.0 233536 10188 ?        Sl   10:38   0:00 /opt/mongodb/bin/mongos --config /data/mongodb/conf/ms0/mongos.conf

五、初始化副本集

配置副本集的好处是为了高可用，配置单节点是我自己为了节省时间，后续添加节点和副本集的操作一样，分片的配置不需要修改,在任何一个节点执行，这里在node1上执行

分片一：

1$ mongo --port 27010
2MongoDB shell version: 3.0.6
3connecting to: 127.0.0.1:27010/test
4> use admin
5switched to db admin
6> cfg={_id:"sh0", members:[ {_id:0,host:"172.30.2.201:27010"}, {_id:1,host:"172.30.2.202:27010"}, {_id:2,host:"172.30.2.203:27010"} ] }
7{
8        "_id" : "sh0",
9        "members" : [
10                {
11                        "_id" : 0,
12                        "host" : "172.30.2.201:27010"
13                },
14                {
15                        "_id" : 1,
16                        "host" : "172.30.2.202:27010"
17                },
18                {
19                        "_id" : 2,
20                        "host" : "172.30.2.203:27010"
21                }
22        ]
23}
24> rs.initiate( cfg );
25{ "ok" : 1 }
26sh0:OTHER> rs.status()
27{
28        "set" : "sh0",
29        "date" : ISODate("2015-10-23T05:33:31.920Z"),
30        "myState" : 1,
31        "members" : [
32                {
33                        "_id" : 0,
34                        "name" : "172.30.2.201:27010",
35                        "health" : 1,
36                        "state" : 1,
37                        "stateStr" : "PRIMARY",
38                        "uptime" : 270,
39                        "optime" : Timestamp(1445578404, 1),
40                        "optimeDate" : ISODate("2015-10-23T05:33:24Z"),
41                        "electionTime" : Timestamp(1445578408, 1),
42                        "electionDate" : ISODate("2015-10-23T05:33:28Z"),
43                        "configVersion" : 1,
44                        "self" : true
45                },
46                {
47                        "_id" : 1,
48                        "name" : "172.30.2.202:27010",
49                        "health" : 1,
50                        "state" : 5,
51                        "stateStr" : "STARTUP2",
52                        "uptime" : 7,
53                        "optime" : Timestamp(0, 0),
54                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
55                        "lastHeartbeat" : ISODate("2015-10-23T05:33:30.289Z"),
56                        "lastHeartbeatRecv" : ISODate("2015-10-23T05:33:30.295Z"),
57                        "pingMs" : 1,
58                        "configVersion" : 1
59                },
60                {
61                        "_id" : 2,
62                        "name" : "172.30.2.203:27010",
63                        "health" : 1,
64                        "state" : 5,
65                        "stateStr" : "STARTUP2",
66                        "uptime" : 7,
67                        "optime" : Timestamp(0, 0),
68                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
69                        "lastHeartbeat" : ISODate("2015-10-23T05:33:30.289Z"),
70                        "lastHeartbeatRecv" : ISODate("2015-10-23T05:33:30.293Z"),
71                        "pingMs" : 1,
72                        "configVersion" : 1
73                }
74        ],
75        "ok" : 1
76}
77sh0:PRIMARY>
78bye

分片二：

1$ mongo --port 27011
2MongoDB shell version: 3.0.6
3connecting to: 127.0.0.1:27011/test
4> use admin
5switched to db admin
6> cfg={_id:"sh1", members:[ {_id:0,host:"172.30.2.201:27011"}, {_id:1,host:"172.30.2.202:27011"}, {_id:2,host:"172.30.2.203:27011"} ] }
7{
8        "_id" : "sh1",
9        "members" : [
10                {
11                        "_id" : 0,
12                        "host" : "172.30.2.201:27011"
13                },
14                {
15                        "_id" : 1,
16                        "host" : "172.30.2.202:27011"
17                },
18                {
19                        "_id" : 2,
20                        "host" : "172.30.2.203:27011"
21                }
22        ]
23}
24> rs.initiate( cfg );
25{ "ok" : 1 }
26sh1:OTHER> rs.status();
27{
28        "set" : "sh1",
29        "date" : ISODate("2015-10-23T05:36:02.365Z"),
30        "myState" : 1,
31        "members" : [
32                {
33                        "_id" : 0,
34                        "name" : "172.30.2.201:27011",
35                        "health" : 1,
36                        "state" : 1,
37                        "stateStr" : "PRIMARY",
38                        "uptime" : 406,
39                        "optime" : Timestamp(1445578557, 1),
40                        "optimeDate" : ISODate("2015-10-23T05:35:57Z"),
41                        "electionTime" : Timestamp(1445578561, 1),
42                        "electionDate" : ISODate("2015-10-23T05:36:01Z"),
43                        "configVersion" : 1,
44                        "self" : true
45                },
46                {
47                        "_id" : 1,
48                        "name" : "172.30.2.202:27011",
49                        "health" : 1,
50                        "state" : 5,
51                        "stateStr" : "STARTUP2",
52                        "uptime" : 5,
53                        "optime" : Timestamp(0, 0),
54                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
55                        "lastHeartbeat" : ISODate("2015-10-23T05:36:01.168Z"),
56                        "lastHeartbeatRecv" : ISODate("2015-10-23T05:36:01.175Z"),
57                        "pingMs" : 0,
58                        "configVersion" : 1
59                },
60                {
61                        "_id" : 2,
62                        "name" : "172.30.2.203:27011",
63                        "health" : 1,
64                        "state" : 5,
65                        "stateStr" : "STARTUP2",
66                        "uptime" : 5,
67                        "optime" : Timestamp(0, 0),
68                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
69                        "lastHeartbeat" : ISODate("2015-10-23T05:36:01.167Z"),
70                        "lastHeartbeatRecv" : ISODate("2015-10-23T05:36:01.172Z"),
71                        "pingMs" : 0,
72                        "configVersion" : 1
73                }
74        ],
75        "ok" : 1
76}
77sh1:PRIMARY>
78bye

六、配置分片

1$ mongo --port 30000
2MongoDB shell version: 3.0.6
3connecting to: 127.0.0.1:30000/test
4mongos> use admin;
5switched to db admin
6mongos> sh.addShard("sh0/172.30.2.201:27010,172.30.2.202:27010,172.30.2.203:27010");
7{ "shardAdded" : "sh0", "ok" : 1 }
8mongos> sh.addShard("sh1/172.30.2.201:27011,172.30.2.202:27011,172.30.2.203:27011");
9{ "shardAdded" : "sh1", "ok" : 1 }
10mongos> use mydb;
11switched to db mydb
12mongos> db.createCollection("test");
13{
14        "ok" : 1,
15        "$gleStats" : {
16                "lastOpTime" : Timestamp(1444358911, 1),
17                "electionId" : ObjectId("56172a4bc03d9b1667f8e928")
18        }
19}
20mongos> sh.enableSharding("mydb");
21{ "ok" : 1 }
22mongos> sh.shardCollection("mydb.test", {"_id":1});
23{ "collectionsharded" : "mydb.test", "ok" : 1 }
24mongos> sh.status();
25--- Sharding Status ---
26  sharding version: {
27        "_id" : 1,
28        "minCompatibleVersion" : 5,
29        "currentVersion" : 6,
30        "clusterId" : ObjectId("561728b4030ea038bcb57fa0")
31}
32  shards:
33        {  "_id" : "sh0",  "host" : "sh0/172.30.2.201:27010,172.30.2.202:27010,172.30.2.203:27010" }
34        {  "_id" : "sh1",  "host" : "sh1/172.30.2.201:27011,172.30.2.202:27011,172.30.2.203:27011" }
35  balancer:
36        Currently enabled:  yes
37        Currently running:  no
38        Failed balancer rounds in last 5 attempts:  0
39        Migration Results for the last 24 hours:
40                No recent migrations
41  databases:
42        {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
43        {  "_id" : "mydb",  "partitioned" : true,  "primary" : "sh0" }
44                mydb.test
45                        shard key: { "_id" : 1 }
46                        chunks:
47                                sh0     1
48                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : sh0 Timestamp(1, 0)

可见分片已经配置完成了

七、添加开机启动项

1$ sudo vim /etc/rc.local
2ulimit -SHn 65535
3/opt/mongodb/bin/mongod --config /data/mongodb/conf/sh0/mongodb.conf
4/opt/mongodb/bin/mongod --config /data/mongodb/conf/sh1/mongodb.conf
5/opt/mongodb/bin/mongod --config /data/mongodb/conf/cf0/config.conf
6/opt/mongodb/bin/mongos --config /data/mongodb/conf/ms0/mongos.conf

八、备注

    虽然也是3台机器，使用分片的好处是可以把两个分片的primary设置在不同的节点，这个可以分摊单节点的压力，当然有更多机器就可以把分片放到不同机器上。