K8S_应用_部署MongoDB复制集


目录:

MongoDB集群部署模式

mongodb采用副本集方式部署(3副本 1primary 2 secondary)

使用kubernetes的mongo集群应考虑需要连接mongo的服务均部署在kubernetes内部,才能通过coreDNS域名或端口访问。

整体架构如下图所示:

使用statefulset模式部署mongod集群具体实现

集群创建mongo service account

$ cat mongo-rbac.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: mongo
  namespace: mongo
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: read-pod-service-endpoint
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - services
  - endpoints
  verbs:
  - get
  - list
  - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: system:serviceaccount:default:mongo
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: read-pod-service-endpoint
subjects:
- kind: ServiceAccount
  name: mongo
  namespace: mongo

创建pvc存储storageclass

$ cat mongo-storageclass.yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: mongo-data
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
# provisioner:[该组件使用现有的 NFS 服务器配置 Kubernetes 持久卷]

此时在后台能看到storageclass配置信息:

通statefulset创建mongo集群,并使用复制集方式实现高可用配置

$ cat mongo-statefulset.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: mongo
  namespace: mongo
  labels:
    name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  clusterIP: None
  selector:
    role: mongo
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongo
  namespace: mongo
spec:
  serviceName: "mongo"
  replicas: 3
  selector:
    matchLabels:
      role: mongo
  template:
    metadata:
      labels:
        role: mongo
        environment: test
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: role
                  operator: In
                  values:
                  - mongo
              topologyKey: kubernetes.io/hostname
      serviceAccountName: mongo
      automountServiceAccountToken: true
      terminationGracePeriodSeconds: 30
      containers:
      - name: mongo
        image: 192.168.254.29:8080/library/mongo:4.4
        command:
        - mongod
        args:
        - "--replSet=rs0"
        - "--bind_ip=0.0.0.0"
        ports:
        - containerPort: 27017
        resources:
          requests:
            cpu: 250m
            memory: 200M  
          limits:
            cpu: 1
            memory: 1024M
        volumeMounts:
        - name: mongodb-persistent-storage-claim
          mountPath: /data/db
      - name: mongo-sidecar  # sidecar模式:将应用程序的组件部署到单独的进程或容器中,以提供隔离和封装
        image: 192.168.254.29:8080/library/k8s-mongo-sidecar:latest
        env:
        - name: KUBERNETES_POD_LABELS
          value: "role=mongo,environment=test"
        - name: KUBERNETES_SERVICE_NAME
          value: "mongo"
  volumeClaimTemplates:
  - metadata:
      name: mongodb-persistent-storage-claim
    spec:
      storageClassName: mongo-data
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 10Gi

查看statefulset资源创建情况

主从读写验证

进入主节点,测试数据写入

进入从节点,测试数据读取

查看此时从节点无法读取集合数据,因为当前从节点只是备份节点,不是slave,无法读写数据。 默认情况下从节点无读写权限,需手动进行设置,配置为主从复制,读写分离。 进入从节点,输入rs.slaveOk()命令,设置从节点读权限

服务暴露

通过master进入mongo服务,查看Primary,创建service

---
apiVersion: v1
kind: Service
metadata:
  name: mongo-cs
  namespace: mongo
  labels:
    name: mongo
spec:
  type: NodePort  # 主要让外部能登录到mongo集群查看,若无需外部连接可移除
  ports:
    - port: 27017
      targetPort: 27017
      nodePort: 30602
  selector:
    statefulset.kubernetes.io/pod-name: mongo-2  # 登录mongo查看primary

客户端连接

应用程序连接Mongo的uri规范: mongodb://[IP]:[Port],[IP]:[Port],[IP]:[Port]/[datebases]

众所周知,在Kubernetes中Pod的IP地址是不固定的,而通过statefulset部署下的服务名是永远保持不变的,所以应用程序通过K8S的服务名进行连接。

k8s的服务名规范:<pod_name>.<service_name>.<namespace>.svc.cluster.local

所以应用程序可以通过mongodb://mongo-0.mongo.mongo.svc.cluster.local:27017,mongo-1.mongo.mongo.svc.cluster.local:27017,mongo-2.mongo.mongo.svc.cluster.local:27017/?replicaSet=rs0)来连接到mongo集群