Etcd 是一个可靠的分布式键值存储, 常用于分布式系统关键数据的存储；而 etcdadm 是一个用于操作 etcd 集群的命令行工具，它可以轻松创建集群、向现有集群添加成员、从现有集群中删除成员等操作；其使用方式类似 kubeadm, 即主要操作流程为: 先启动第一个集群节点，后续节点直接 join 即可

测试环境

节点主机名 节点 IP 地址 系统版本 etcd 版本 etcdadm 版本 c7 192.168.31.37 CentOS 7.9.2009 ( 5.4.180-1.el7 ) V3.5.5 V0.1.5 c8 192.168.31.38 同上 同上 同上 c9 192.168.31.39 同上 同上 同上

安装 etcdadm

1、预编译二进制安装

wget  etcdadm-linux-amd64 /usr/local/bin/etcdadmchmod +x /usr/local/bin/etcdadmscp /usr/local/bin/etcdadm 192.168.31.{38,39}:/usr/local/bin/

2、各节点系统防火墙放行端口 2379，2380

firewall-cmd --add-port=2379/tcpfirewall-cmd --add-port=2380/tcp

1、初始化第一个 etcd 集群节点

etcdadm init \--version "3.5.5" \--init-system "systemd" \--install-dir "/opt/bin/" \--certs-dir "/etc/etcd/pki" \--data-dir "/var/lib/etcd" \--release-url ";# 主要选项解析--version      # 指定部署的 etcd 版本--init-system  # 设置 etcd 进程管理方式，默认 systemd，取值 kubelet 时，则以容器方法运行 etcd 进程--install-dir  # etcd 二进制程序安装目录

2、etcdadm init 初始化过程解析

# 下载解压、安装二进制文件 etcd、etcdctl2022-10-20 14:26:12.781166 I | [install] Artifact not found in cache. Trying to fetch from upstream: [0000] [install] Downloading & installing etcd  from 3.5.5 to /var/cache/etcdadm/etcd/v3.5.5INFO[0000] [install] downloading etcd from  to /var/cache/etcdadm/etcd/v3.5.5/etcd-v3.5.5-linux-amd64.tar.gzINFO[0009] [install] extracting etcd archive /var/cache/etcdadm/etcd/v3.5.5/etcd-v3.5.5-linux-amd64.tar.gz to /tmp/etcd641204404INFO[0009] [install] verifying etcd 3.5.5 is installed in /opt/bin/# 生成一个自签名的 CA 证书及私钥INFO[0001] [certificates] creating PKI assetsINFO[0001] creating a self signed etcd CA certificate and key files[certificates] Generated ca certificate and key.> /etc/etcd/pki/ca.crt> /etc/etcd/pki/ca.key# 生成一个 server 证书及私钥INFO[0001] creating a new server certificate and key files for etcd[certificates] Generated server certificate and key.[certificates] server serving cert is signed for DNS names [c7] and IPs [192.168.31.37 127.0.0.1]# > /etc/etcd/pki/server.crt# > /etc/etcd/pki/server.key# 生成一个 peer 证书及私钥INFO[0001] creating a new certificate and key files for etcd peering[certificates] Generated peer certificate and key.[certificates] peer serving cert is signed for DNS names [c7] and IPs [192.168.31.37]# > /etc/etcd/pki/peer.crt# > /etc/etcd/pki/peer.key# 生成一个用于 etcdctl 的 client 证书及私钥INFO[0001] creating a new client certificate for the etcdctl[certificates] Generated etcdctl-etcd-client certificate and key.# > /etc/etcd/pki/etcdctl-etcd-client.crt# > /etc/etcd/pki/etcdctl-etcd-client.key# 生成一个用于 k8s apiserver 调用 etcd 时的 client 证书及私钥INFO[0002] creating a new client certificate for the apiserver calling etcd[certificates] Generated apiserver-etcd-client certificate and key.[certificates] valid certificates and keys now exist in "/etc/etcd/pki"# > /etc/etcd/pki/apiserver-etcd-client.crt# > /etc/etcd/pki/apiserver-etcd-client.key# 检查本地 etcd 端点是否健康INFO[0003] [health] Checking local etcd endpoint healthINFO[0003] [health] Local etcd endpoint is healthy# 复制 CA cert/key 到其它 etcd 节点，并在其它 etcd 节点运行 etcdadm join 命令， 将其它 etcd 节点加入集群INFO[0003] To add another member to the cluster, copy the CA cert/key to its certificate dir and run:INFO[0003]  etcdadm join 

3、向其它节点分发 CA 根证书及私钥

ssh root@192.168.31.38 "mkdir /etc/etcd/pki/"scp -r /etc/etcd/pki/{ca.crt,ca.key} 192.168.31.38:/etc/etcd/pki/ssh root@192.168.31.39 "mkdir /etc/etcd/pki/"scp -r /etc/etcd/pki/{ca.crt,ca.key} 192.168.31.39:/etc/etcd/pki/

若当前主机无法下载，可提前将 etcd 二进制程序包存放在如下路径: /var/cache/etcdadm/etcd/v3.5.5/etcd-v3.5.5-linux-amd64.tar.gz

1、添加节点 192.168.31.38

etcdadm join  \--version "3.5.5" \--init-system "systemd" \--install-dir "/opt/bin/"   \--certs-dir  "/etc/etcd/pki" \--data-dir "/var/lib/etcd"  \--release-url ";

2、添加节点 192.168.31.39

etcdadm join  \--version "3.5.5" \--init-system "systemd" \--install-dir "/opt/bin/"   \--certs-dir  "/etc/etcd/pki" \--data-dir "/var/lib/etcd"  \--release-url ";

1、用于 Etcd Server 的环境变量配置 /etc/etcd/etcd.env

ETCD_NAME=c7# Initial cluster configurationETCD_INITIAL_CLUSTER=c7= Peer configurationETCD_INITIAL_ADVERTISE_PEER_URLS= Client/server configurationETCD_ADVERTISE_CLIENT_URLS= OtherETCD_DATA_DIR=/var/lib/etcdETCD_STRICT_RECONFIG_CHECK=trueGOMAXPROCS=8# Logging configuration# Profiling/metrics

2、Etcd Server 启动脚本

# cat /etc/systemd/system/etcd.service[Unit]Description=etcdDocumentation=[Service]EnvironmentFile=/etc/etcd/etcd.envExecStart=/opt/bin/etcdType=notifyTimeoutStartSec=0Restart=on-failureRestartSec=5sLimitNOFILE=65536Nice=-10IOSchedulingClass=best-effortIOSchedulingPriority=2MemoryLow=200M[Install]WantedBy=multi-user.target

1、用于 etcdctl 的环境变量配置 /etc/etcd/etcdctl.env

export ETCDCTL_API=3export ETCDCTL_CACERT=/etc/etcd/pki/ca.crtexport ETCDCTL_CERT=/etc/etcd/pki/etcdctl-etcd-client.crtexport ETCDCTL_KEY=/etc/etcd/pki/etcdctl-etcd-client.keyexport ETCDCTL_DIAL_TIMEOUT=3s

2、脚本 etcdctl.sh 是对 etcdctl 命令的简单包装，其用法与 etcdctl 一致

cat /opt/bin/etcdctl.sh#!/usr/bin/env shif ! [ -r "/etc/etcd/etcdctl.env" ]; then        echo "Unable to read the etcdctl environment file '/etc/etcd/etcdctl.env'. The file must exist, and this wrapper must be run as root."        exit 1fi. "/etc/etcd/etcdctl.env"   # 相当于 source 该环境变量配置文件"/opt/bin/etcdctl" "$@"     # $@ 表示脚本 etcdctl.sh 的命令行参数

# 查看命令行 init 或 join 的帮助信息etcdadm init|join --help# 从 etcd 集群移除当前节点etcdadm reset# 查看集群节点成员/opt/bin/etcdctl.sh member list# > 19fc11a542653f62, started, c9, , , false# > 9a246c6786d36273, started, c7, , , false# > a509d3d8e8aa4911, started, c8, , , false# 查看当前节点是否正常/opt/bin/etcdctl.sh endpoint health# 127.0.0.1:2379 is healthy: successfully committed proposal: took = 17.112587ms# 查看当前节点状态/opt/bin/etcdctl.sh endpoint status# > 127.0.0.1:2379, 9a246c6786d36273, 3.5.5, 20 kB, true, false, 3, 10, 10,