mongodb分片集群的介绍和搭建

mongodb的分片集群由3部分组成,Router,ConfigServer,shard。

Router:负责路由,应用请求到Router,会去找到ConfigServer,获取到配置信息,然后会把数据存到Shard。
ConfigServer:记录的是一些节点,配置,数据分布信息,类似linux /etc/目录,把etc拿到专门的一个节点去存储,目前这种产品有zookeeper,etcd。需要注意的是config server必须要是普通节点不支持arbiter。
Shard: 就是可以1主1从1arbiter,也可以是1主2从。shard注意存储的就是数据。

另外mongodb是支持自动分片的,不过一般生产上还是自己定义策略。

分片集群搭建

环境

这里用10个实例来做演示:
端口为30005-30014

mongos

30005

configserver

  • 1主2从
  • 端口30006,30007,30008
  • 复制集名字为configRepSet。

shard节点

shard1
  • 1主2从,其中1个为arbiter(仲裁节点)
  • 端口30009,30010,30011
  • 复制集名字为s1。
shard2
  • 1主2从,其中1个为arbiter(仲裁节点)
  • 端口300012,30013,30014
  • 复制集名字为s2。

搭建

下面所有命令直接复制命令行即可
首先创建所需目录

for((i=30005;i<=30014;i++));do mkdir -p /opt/modules/mongodb/$i/{conf,log,data};done

搭建shard

搭建第一组复制集s1 09-11(1主1从1arbiter)

cat > /opt/modules/mongodb/30009/conf/mongod.conf << EOF
systemLog:
    destination: file
    path: /opt/modules/mongodb/30009/log/mongodb.log
    logAppend: true
storage:
    journal:
        enabled: true
    dbPath: "/opt/modules/mongodb/30009/data"
    directoryPerDB: true
    wiredTiger:
        engineConfig:
            cacheSizeGB: 1
            directoryForIndexes: true
        collectionConfig:
            blockCompressor: zlib
        indexConfig:
            prefixCompression: true
processManagement:
    fork: true
net:
    port: 30009
    bindIp: 192.168.3.66,127.0.0.1 
replication:
    oplogSizeMB: 2048
    replSetName: s1
sharding:
    clusterRole: shardsvr    
EOF

# 复制到多个文件夹
\cp /opt/modules/mongodb/30009/conf/mongod.conf /opt/modules/mongodb/30010/conf/
\cp /opt/modules/mongodb/30009/conf/mongod.conf /opt/modules/mongodb/30011/conf/ 

sed -i 's#30009#30010#g' /opt/modules/mongodb/30010/conf/mongod.conf
sed -i 's#30009#30011#g' /opt/modules/mongodb/30011/conf/mongod.conf

搭建s2 12-14(1主1从1arbiter)

cat > /opt/modules/mongodb/30012/conf/mongod.conf << EOF
systemLog:
    destination: file
    path: /opt/modules/mongodb/30012/log/mongodb.log
    logAppend: true
storage:
    journal:
        enabled: true
    dbPath: "/opt/modules/mongodb/30012/data"
    directoryPerDB: true
    wiredTiger:
        engineConfig:
            cacheSizeGB: 1
            directoryForIndexes: true
        collectionConfig:
            blockCompressor: zlib
        indexConfig:
            prefixCompression: true
processManagement:
    fork: true
net:
    port: 30012
    bindIp: 192.168.3.66,127.0.0.1 
replication:
    oplogSizeMB: 2048
    replSetName: s2
sharding:
    clusterRole: shardsvr    
EOF

# 复制到多个文件夹
\cp /opt/modules/mongodb/30012/conf/mongod.conf /opt/modules/mongodb/30013/conf/
\cp /opt/modules/mongodb/30012/conf/mongod.conf /opt/modules/mongodb/30014/conf/ 

sed -i 's#30012#30013#g' /opt/modules/mongodb/30013/conf/mongod.conf
sed -i 's#30012#30014#g' /opt/modules/mongodb/30014/conf/mongod.conf

启动所有节点

for((i=30009;i<=30014;i++));do mongod -f /opt/modules/mongodb/$i/conf/mongod.conf;done

配置复制集

# 配置s1
# 进入mongo
mongo --port 30009 admin
config = {_id: 's1', members:[
            {_id: 0, host: "192.168.3.66:30009"},
            {_id: 1, host: "192.168.3.66:30010"},
            {_id: 2, host: "192.168.3.66:30011","arbiterOnly":true}]
        }
rs.initiate(config)

# 配置s2
mongo --port 30012 admin
config = {_id: 's2', members:[
            {_id: 0, host: "192.168.3.66:30012"},
            {_id: 1, host: "192.168.3.66:30013"},
            {_id: 2, host: "192.168.3.66:30014","arbiterOnly":true}]
        }
rs.initiate(config)

搭建ConfigServer

设置配置文件

cat > /opt/modules/mongodb/30006/conf/mongod.conf << EOF
systemLog:
    destination: file
    path: /opt/modules/mongodb/30006/log/mongodb.log
    logAppend: true
storage:
    journal:
        enabled: true
    dbPath: "/opt/modules/mongodb/30006/data"
    directoryPerDB: true
    wiredTiger:
        engineConfig:
            cacheSizeGB: 1
            directoryForIndexes: true
        collectionConfig:
            blockCompressor: zlib
        indexConfig:
            prefixCompression: true
processManagement:
    fork: true
net:
    port: 30006
    bindIp: 192.168.3.66,127.0.0.1 
replication:
    oplogSizeMB: 2048
    replSetName: configRepSet
sharding:
    clusterRole: configsvr
EOF

# 复制到多个文件夹
\cp /opt/modules/mongodb/30006/conf/mongod.conf /opt/modules/mongodb/30007/conf/
\cp /opt/modules/mongodb/30006/conf/mongod.conf /opt/modules/mongodb/30008/conf/ 

sed -i 's#30006#30007#g' /opt/modules/mongodb/30007/conf/mongod.conf
sed -i 's#30006#30008#g' /opt/modules/mongodb/30008/conf/mongod.conf

启动所有节点

for((i=30006;i<=30008;i++));do mongod -f /opt/modules/mongodb/$i/conf/mongod.conf;done

配置复制集

mongo --port 30006 admin
config = {_id: 'configRepSet', members:[
            {_id: 0, host: "192.168.3.66:30006"},
            {_id: 1, host: "192.168.3.66:30007"},
            {_id: 2, host: "192.168.3.66:30008"}]
        }
rs.initiate(config)

Router(mongos)

注意mongos是没有数据目录。

cat > /opt/modules/mongodb/30005/conf/mongos.conf << EOF
systemLog:
    destination: file
    path: "/opt/modules/mongodb/30005/log/mongos.log"
    logAppend: true    
processManagement:
    fork: true
net:
    port: 30005
    bindIp: 192.168.3.66,127.0.0.1
sharding:
    configDB: configRepSet/192.168.3.66:30006,192.168.3.66:30007,192.168.3.66:30008
EOF

启动mongos

mongos -f /opt/modules/mongodb/30005/conf/mongos.conf

分片集群添加节点

# 连接到mongos
mongo 192.168.3.66:30005/admin
# 添加分片
db.runCommand( {addshard:"s1/192.168.3.66:30009,192.168.3.66:30010,192.168.3.66:30011",namme:"shard1"} )
db.runCommand( {addshard:"s2/192.168.3.66:30012,192.168.3.66:30013,192.168.3.66:30014",namme:"shard2"} )
# 列出分片
db.runCommand( { listshards: 1 } )
# 查看状态
sh.status()

range分片策略

需要注意,range分片有个特点,如果数据量小的话,不足以占用一个chunk的话,只会存在一个节点上。所以下面测试做一个大的数据量插入。

首先连接的mongos节点

mongo 192.168.3.66:30005/admin

开启test库的分片功能

> db.runCommand( { enablesharding:"pay" } )

指定分片建(指定以哪一列的值进行分片条件),对集合进行分片

创建索引
> use pay
> db.vast.ensureIndex( { id:1 } )

开启分片

> use adimn
> db.runCommand( { shardcollection:"pay.success_log",key: {id:1} } )

分片验证

> use pay
> for(i=1;i<10000;i++){ db.success_log.insert({"id":i,"name":"shenzheng","age":70,"date":new Date()}); }
> db.success_log.stats()

hash分片策略

连接mongos节点开启pay库的分片功能

mongo --port 30005/admin
> use admin
> db.runCommand( { enablesharding:"pay" } )

对pay_log表建立hash索引

> use pay
> db.pay_log.ensureIndex( {id:"hashed"} )

开启分片

use admin
> sh.shardCollection("pay.pay_log",{id:"hashed"})

录入数据测试

> use pay_log
> for(i=1;i<100000;i++){ db.pay_log.insert({"id":i,"name":"shenzheng","age":70,"date":new Date()}); }

分别查看s1和s2节点的数据

# 查看s1数据
mongos --port 30009
use pay
db.pay_log.count()