前言

部署mysql之前我们需要先了解一个概念有状态服务。这是一种特殊的服务，简单的归纳下就是会产生需要持久化的数据，并且有很强的I/O需求，且重启需要依赖上次存储到磁盘的数据。如典型的mysql，kafka，zookeeper等等。

在我们有比较优秀的商业存储的前提下，灰常推荐使用有状态服务进行部署，计算和存储分离那是相当的爽的。在实际生产中如果没有这种存储，才有localPV也是不错的选择，当然local pv其实呢和hostPath是一样的。当然我们在开发测试环境也是可以自己搭建一套简单的如NFS服务，来享受存储和计算分离的爽快感。

kubernetes中定义一种了资源类型Stateful Service即有状态服务，有状态服务需要的持久化数据动态绑定我们可以利用存储的API PersistentVolume（PV）和PersistentVolumeClaim（PVC）来进行需要的相关数据的绑定和存储。

PV & PVC

PV就好比是一个仓库，我们需要先购买一个仓库，即定义一个PV存储服务，例如CEPH,NFS,Local Hostpath等等。PVC就好比租户，pv和pvc是一对一绑定的，挂载到POD中，一个pvc可以被多个pod挂载。大致一个流程如下，可以从这里,以及官网看到更多的额关于PV的细节

创建PV --> 创建PVC --> 绑定 --> 可写入数据

有了这个理解之后，我们接下来实战一下（mysql(pvc)+NFS（PV））

NFS Server

首先我们需要创建一个nfs server。我从hub.docker找到一个nfs-server。下面我们将它部署到kubernetes中。

部署nfs到kubernetes

# nfs-server.yamlapiVersion: v1kind: Servicemetadata:  name: nfs-service  labels:    app: nfs-servicespec:  ports:    - port: 2049      name: nfs-service  clusterIP: 10.96.0.14  selector:    app: nfs-service---apiVersion: apps/v1kind: Deploymentmetadata:  name: nfs-servicespec:  strategy:    type: Recreate  selector:    matchLabels:      app: nfs-service  template:    metadata:      labels:        app: nfs-service    spec:      # 采用node选择器      nodeName: node1      containers:        - name: nfs-service          image: itsthenetwork/nfs-server-alpine:latest          imagePullPolicy: IfNotPresent          securityContext:            privileged: true            capabilities:              add:                - SYS_ADMIN                - SETPCAP          livenessProbe:            tcpSocket:              port: 2049            timeoutSeconds: 5          readinessProbe:            tcpSocket:              port: 2049            timeoutSeconds: 5          ports:            - containerPort: 2049          env:            - name: SHARED_DIRECTORY              value: /nfsshare          volumeMounts:            - mountPath: /nfsshare              name: nfsshare      volumes:        - name: nfsshare          hostPath:            path: /nfsshare            type: DirectoryOrCreate

部署到集群

script1kubectl apply -f nfs-server.yaml

查看日志

scriptroot@server1:~# kubectl logs -f deploy/nfs-serviceStarting rpcbind...Displaying rpcbind status...   program version netid     address                service    owner    100000    4    tcp       0.0.0.0.0.111          -          superuser    100000    3    tcp       0.0.0.0.0.111          -          superuser    100000    2    tcp       0.0.0.0.0.111          -          superuser    100000    4    udp       0.0.0.0.0.111          -          superuser    100000    3    udp       0.0.0.0.0.111          -          superuser    100000    2    udp       0.0.0.0.0.111          -          superuser    100000    4    local     /var/run/rpcbind.sock  -          superuser    100000    3    local     /var/run/rpcbind.sock  -          superuserStarting NFS in the background...rpc.nfsd: knfsd is currently downrpc.nfsd: Writing version string to kernel: -2 -3 +4 +4.1 +4.2rpc.nfsd: Created AF_INET TCP socket.Exporting File System...exporting *:/nfsshare/nfsshare       <world>Starting Mountd in the background...TheseStartup successful.

安装nfs-client。

CentOS

script1sudo yum install nfs-utils

Ubuntu

script1sudo apt-get install nfs-common -y

挂载到node的目录上

script1sudo mount -t nfs -o vers=4,minorversion=0,noresvport 10.96.0.14:/ /nfs

向挂载的NFS木写入数据，

scriptecho 1 > /nfs/1.txt

前往node1查看，看到如下结果表示NFS服务器搭建成功。

scriptroot@node1:/nfsshare# ls1.txtroot@node1:/nfsshare# cat 1.txt 1

搭建完了nfs之后，我们就可以创建PV了

部署pv

apiVersion: v1kind: PersistentVolumemetadata:  name: mysql-static-pvspec:  capacity:    storage: 10Gi  storageClassName: mysql-scn  #ReadWriteOnce - 卷可以由单个节点以读写方式挂载  #ReadOnlyMany - 卷可以由许多节点以只读方式挂载  #ReadWriteMany - 卷可以由许多节点以读写方式挂载  accessModes:    - ReadWriteOnce  #Retain，不清理, 保留 Volume（需要手动清理）  #Recycle，删除数据，即 rm -rf /thevolume/*（只有 NFS 和 HostPath 支持）  #Delete，删除存储资源，比如删除 AWS EBS 卷（只有 AWS EBS, GCE PD, Azure Disk 和 Cinder 支持）  persistentVolumeReclaimPolicy: Retain  nfs:    path: /    server: 10.96.0.14  mountOptions:    - vers=4    - minorversion=0    - noresvport

部署PV到集群中

script$ kubectl apply -f mysql-pv.yaml persistentvolume/mysql-static-pv created

查看PV状态，看到status为Available表示PV创建成功。

script$ kubectl get pvNAME              CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGEmysql-static-pv   10Gi       RWO            Retain           Available           mysql-scn               3s

scriptkind: PersistentVolumeClaimapiVersion: v1metadata:  name: mysql-static-pvcspec:  #RWO - ReadWriteOnce  #ROX - ReadOnlyMany  #RWX - ReadWriteMany  accessModes:    - ReadWriteOnce  storageClassName: mysql-scn  resources:    requests:      storage: 10Gi

部署到集群中

script$ kubectl apply -f mysql-static-pvc.yaml persistentvolumeclaim/mysql-static-pvc created

查看状态，看到STATUS=Bound表示PVC和PV绑定成功了。可以看到我们有一个名称为mysql-static-pvc的pvc可以用了。

scriptfreemandeMacBook-Pro:pv-pvc freeman$ kubectl get pvcNAME               STATUS   VOLUME            CAPACITY   ACCESS MODES   STORAGECLASS   AGEmysql-static-pvc   Bound    mysql-static-pv   10Gi       RWO            mysql-scn      4s

# mysql-server.yamlapiVersion: v1kind: Servicemetadata:  name: mysql  labels:    app: mysqlspec:  ports:    - port: 3306      name: mysql  clusterIP: 10.96.0.15  selector:    app: mysql---apiVersion: apps/v1kind: Deploymentmetadata:  name: mysqlspec:  strategy:    type: Recreate  selector:    matchLabels:      app: mysql  template:    metadata:      labels:        app: mysql    spec:      containers:        - name: mysql          image: mysql/mysql-server:8.0.17          imagePullPolicy: IfNotPresent          livenessProbe:            tcpSocket:              port: 3306            timeoutSeconds: 5          readinessProbe:            tcpSocket:              port: 3306            timeoutSeconds: 5          ports:            - containerPort: 3306          volumeMounts:            - mountPath: /var/lib/mysql              name: mysql-data              readOnly: false          env:            - name: MYSQL_ROOT_PASSWORD              value: "root"      volumes:        - name: mysql-data          persistentVolumeClaim:            # 这里名称需要和pvc一一对应            claimName: mysql-static-pvc

scriptkubectl apply -f mysql-server.yaml

查看日志

$ kubectl logs -f deploy/mysql...[Entrypoint] Starting MySQL 8.0.17-1.1.122019-08-22T09:44:49.137554Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.17) starting as process 12019-08-22T09:44:49.754020Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.2019-08-22T09:44:49.795415Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.17'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server - GPL.2019-08-22T09:44:49.943852Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock' bind-address: '0.0.0.0' port: 33060

查看运行状态

script$ kubectl get pods -owide | grep mysqlmysql-f475dc6c8-xnb4x                      1/1     Running            0          5m20s   172.224.3.158   node1               <none>           <none>

查看node1 nfs目录下的数据，可以看到mysql的文件都存储到了我们的nfs属猪鸡的文件系统中了。而我们在部署mysql时，是不需要指定node也没有挂载hostpath，这意味着我们的mysql计算是可以运行在任意集群中的noe上。

scriptroot@node1:/nfsshare# ls /nfsshare/ auto.cnf        binlog.index   client-cert.pem   ibdata1       ibtmp1          mysql.ibd         performance_schema   server-cert.pem   undo_001 binlog.000001   ca-key.pem     client-key.pem    ib_logfile0  '#innodb_temp'   mysql.sock        private_key.pem      server-key.pem    undo_002 binlog.000002   ca.pem         ib_buffer_pool    ib_logfile1   mysql           mysql.sock.lock   public_key.pem       sys

我们先看一下目前mysql是运行在哪个节点上，可以看到目前是运行在node1上。

script$ kubectl get pods -owide | grep mysqlmysql-f475dc6c8-xnb4x                      1/1     Running            0          5m20s   172.224.3.158   node1               <none>           <none>

为了方便测试我修改一下mysql的yaml文件加入node选择器,将mysql强行调度到node2运行。

spec.template.spec.nodeName: node2

重新apply

$ kubectl apply -f mysql-server.yaml service/mysql unchangeddeployment.apps/mysql configured

查看mysql是否还运行正常,可以看到运行正常，这就是存储和计算分离的爽点。

script$ kubectl get pods -owide | grep mysqlmysql-6f5944cbcd-nd7c4                     1/1     Running            0          38s     172.224.4.18    node2               <none>           <none>

script$ kubectl get pods | grep mysqlmysql-6f5944cbcd-nd7c4                     1/1     Running            0          7m58s$ kubectl exec -it mysql-6f5944cbcd-nd7c4 bash$ mysql -uroot -proot$ CREATE USER 'root'@'%' IDENTIFIED BY 'root';$ GRANT ALL ON *.* TO 'root'@'%';$ ALTER USER 'root'@'%' IDENTIFIED WITH mysql_native_password BY 'root';$ FLUSH PRIVILEGES;

在开发机进行访问

script$ mysql -uroot -proot -h 10.96.0.15mysql: [Warning] Using a password on the command line interface can be insecure.Welcome to the MySQL monitor.  Commands end with ; or \g.Your MySQL connection id is 136Server version: 8.0.17 MySQL Community Server - GPLCopyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql>

到此我们完成了基于Mysql+nfs的部署工作。