MongoDB集群部署模式
mongodb采用副本集方式部署(3副本 1primary 2 secondary)
使用kubernetes的mongo集群应考虑需要连接mongo的服务均部署在kubernetes内部,才能通过coreDNS域名或端口访问。
整体架构如下图所示:
使用statefulset模式部署mongod集群具体实现
集群创建mongo service account
$ cat mongo-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: mongo
namespace: mongo
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: read-pod-service-endpoint
rules:
- apiGroups:
- ""
resources:
- pods
- services
- endpoints
verbs:
- get
- list
- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: system:serviceaccount:default:mongo
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: read-pod-service-endpoint
subjects:
- kind: ServiceAccount
name: mongo
namespace: mongo
创建pvc存储storageclass
$ cat mongo-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: mongo-data
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
# provisioner:[该组件使用现有的 NFS 服务器配置 Kubernetes 持久卷]
此时在后台能看到storageclass配置信息:
通statefulset创建mongo集群,并使用复制集方式实现高可用配置
$ cat mongo-statefulset.yaml
---
apiVersion: v1
kind: Service
metadata:
name: mongo
namespace: mongo
labels:
name: mongo
spec:
ports:
- port: 27017
targetPort: 27017
clusterIP: None
selector:
role: mongo
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mongo
namespace: mongo
spec:
serviceName: "mongo"
replicas: 3
selector:
matchLabels:
role: mongo
template:
metadata:
labels:
role: mongo
environment: test
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: role
operator: In
values:
- mongo
topologyKey: kubernetes.io/hostname
serviceAccountName: mongo
automountServiceAccountToken: true
terminationGracePeriodSeconds: 30
containers:
- name: mongo
image: 192.168.254.29:8080/library/mongo:4.4
command:
- mongod
args:
- "--replSet=rs0"
- "--bind_ip=0.0.0.0"
ports:
- containerPort: 27017
resources:
requests:
cpu: 250m
memory: 200M
limits:
cpu: 1
memory: 1024M
volumeMounts:
- name: mongodb-persistent-storage-claim
mountPath: /data/db
- name: mongo-sidecar # sidecar模式:将应用程序的组件部署到单独的进程或容器中,以提供隔离和封装
image: 192.168.254.29:8080/library/k8s-mongo-sidecar:latest
env:
- name: KUBERNETES_POD_LABELS
value: "role=mongo,environment=test"
- name: KUBERNETES_SERVICE_NAME
value: "mongo"
volumeClaimTemplates:
- metadata:
name: mongodb-persistent-storage-claim
spec:
storageClassName: mongo-data
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 10Gi
查看statefulset资源创建情况
主从读写验证
进入主节点,测试数据写入
进入从节点,测试数据读取
查看此时从节点无法读取集合数据,因为当前从节点只是备份节点,不是slave,无法读写数据。
默认情况下从节点无读写权限,需手动进行设置,配置为主从复制,读写分离。
进入从节点,输入
rs.slaveOk()
命令,设置从节点读权限
服务暴露
通过master进入mongo服务,查看Primary,创建service
---
apiVersion: v1
kind: Service
metadata:
name: mongo-cs
namespace: mongo
labels:
name: mongo
spec:
type: NodePort # 主要让外部能登录到mongo集群查看,若无需外部连接可移除
ports:
- port: 27017
targetPort: 27017
nodePort: 30602
selector:
statefulset.kubernetes.io/pod-name: mongo-2 # 登录mongo查看primary
客户端连接
应用程序连接Mongo的uri规范: mongodb://[IP]:[Port],[IP]:[Port],[IP]:[Port]/[datebases]
众所周知,在Kubernetes中Pod的IP地址是不固定的,而通过statefulset部署下的服务名是永远保持不变的,所以应用程序通过K8S的服务名进行连接。
k8s的服务名规范:<pod_name>.<service_name>.<namespace>.svc.cluster.local
所以应用程序可以通过mongodb://mongo-0.mongo.mongo.svc.cluster.local:27017,mongo-1.mongo.mongo.svc.cluster.local:27017,mongo-2.mongo.mongo.svc.cluster.local:27017/?replicaSet=rs0)
来连接到mongo集群