Etcd证书过期解决方法

Etcd是Kubernetes核心组件，存放着Kubernetes数据，你想象到的资源对象都保存在里面，比如namespace、pod、configmap等等资源。而Etcd中的证书有效时间却是一定的，一不注意etcd证书过期，我们应该怎么办？以及如何查看etcd证书过期呢？本文主要介绍如何查看etcd是否过期，以及过期了怎么处理。

1 演示环境信息

一台etcd（生产环境一般是三台）

2 怎么查看Etcd证书是否过期

查看etcd证书目录，执行如下指令systemctl status etcd，可以看到挂载目录有/etc/ssl/etcd/ssl，此处为该环境下etcd证书目录。不同环境下，可能etcd证书目录不一样，通过systemctl status etcd获取。如果没有挂载目录，说明此集群的etcd没有证书。

systemctl status etcd
  etcd.service - etcd docker wrapper
   Loaded: loaded (/etc/systemd/system/etcd.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2021-01-26 21:15:20 CST; 33min ago
  Process: 25093 ExecStop=/usr/bin/docker stop etcd1 (code=exited, status=0/SUCCESS)
  Process: 14658 ExecStartPre=/usr/bin/docker rm -f etcd1 (code=exited, status=0/SUCCESS)
 Main PID: 14686 (etcd)
    Tasks: 11
   Memory: 24.5M
   CGroup: /system.slice/etcd.service
           ├─14686 /bin/bash /usr/local/bin/etcd
           └─14688 /usr/bin/docker run --restart=on-failure:5 --env-file=/etc/etcd.env --net=host -v /etc/ssl/certs:/etc/ssl/certs:ro -v /etc/ssl/etcd/ssl...

Jan 26 21:30:23 node1 etcd[14686]: 2021-01-26 13:30:23.825829 I | mvcc: finished scheduled compaction at 7399951 (took 1.585838ms)

以上步骤知道etcd证书目录，进入此目录下，执行如下指令即可查看Etcd证书过期日期。

cd /etc/ssl/etcd/ssl/
[root@node1 ssl]# ls
admin-node1-key.pem  admin-node1.pem  ca.crt  ca.key  ca-key.pem  ca.pem  member-node1-key.pem  member-node1.pem  node-node1-key.pem  node-node1.pem

查看CA证书有效时间
[root@node1 ssl]# openssl x509 -in ca.pem -noout -text | grep 'Not'
            Not Before: Nov 16 13:04:25 2020 GMT
            Not After : Nov 16 13:04:25 2021 GMT
查看CA签发的Etcd证书有效时间，不同方法部署的Etcd，可能名字不一样。
[root@node1 ssl]# openssl x509 -in member-node1.pem -noout -text | grep 'Not'
            Not Before: Nov 16 13:08:36 2020 GMT
            Not After : Nov 16 13:08:36 2021 GMT
[root@node1 ssl]# openssl x509 -in admin-node1.pem -noout -text | grep 'Not'
            Not Before: Nov 16 13:10:43 2020 GMT
            Not After : Nov 16 13:10:43 2021 GMT
[root@node1 ssl]# openssl x509 -in node-node1.pem -noout -text | grep 'Not'
            Not Before: Nov 16 13:12:44 2020 GMT
            Not After : Nov 16 13:12:44 2022 GMT

3 Etcd证书过期处理方法

3.1 CA和Etcd证书是否都过期

通过以上描述方法查看Etcd证书过期时间，如果CA没有过期，可以不执行更新CA证书步骤。

3.2 停止etcd及备份etcd数据，如果是多台机器构建的Etcd，多台机器都需要执行停止

停止Etcd操作
systemctl stop etcd

备份Etcd数据
cd /var/lib
tar -zvcf etcd.tar.gz etcd/

3.3 配置文件，在证书目录创建openssl.conf，最下面一行，如果是三台机器构建的Etcd集群，则三台ip都需要加上。

[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name

[req_distinguished_name]

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[ ssl_client ]
extendedKeyUsage = clientAuth, serverAuth
basicConstraints = CA:FALSE
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
subjectAltName = @alt_names

[ v3_ca ]
basicConstraints = CA:TRUE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
authorityKeyIdentifier=keyid:always,issuer

[alt_names]
DNS.1 = localhost
DNS.2 = etcd.kube-system.svc.cluster.local
DNS.3 = etcd.kube-system.svc
DNS.4 = etcd.kube-system
DNS.5 = etcd
DNS.6 = lb.kubesphere.local
DNS.7 = node1
IP.1 = 127.0.0.1
IP.2 = 192.168.0.3

3.4 生成新的证书

在其中一台Etcd机器上操作，多台情况，将第一台机器新生成的证书拷贝对应的目录下即可。

备份过期证书，如果CA证书没有过期，除了ca.pem和ca-key.pem，其余都备份。如果都过期，则全部备份。
mkdir -p /tmp/test
mv /etc/ssl/etcd/ssl/* /tmp/test/

生成新的证书
CA证书，新的证书为10年，如果没有过期，不需要执行这步。
openssl genrsa -out ca-key.pem 2048 > /dev/null 2>&1
openssl req -x509 -new -nodes -key ca-key.pem -days 3650 -out ca.pem -subj "/CN=etcd-ca" > /dev/null 2>&1

Etcd证书，新的证书为10年,${host}为Etcd主机名，此处为node1。
# Member key
openssl genrsa -out member-${host}-key.pem 2048 > /dev/null 2>&1
openssl req -new -key member-${host}-key.pem -out member-${host}.csr -subj "/CN=etcd-member-${host}" -config openssl.conf > /dev/null 2>&1
openssl x509 -req -in member-${host}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out member-${host}.pem -days 3650 -extensions ssl_client -extfile openssl.conf > /dev/null 2>&1

# Admin key
openssl genrsa -out admin-${host}-key.pem 2048 > /dev/null 2>&1
openssl req -new -key admin-${host}-key.pem -out admin-${host}.csr -subj "/CN=etcd-admin-${host}" > /dev/null 2>&1
openssl x509 -req -in admin-${host}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out admin-${host}.pem -days 3650 -extensions ssl_client  -extfile openssl.conf > /dev/null 2>&1


# Node keys
openssl genrsa -out node-${host}-key.pem 2048 > /dev/null 2>&1
openssl req -new -key node-${host}-key.pem -out node-${host}.csr -subj "/CN=etcd-node-${host}" > /dev/null 2>&1
openssl x509 -req -in node-${host}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out node-${host}.pem -days 3650 -extensions ssl_client  -extfile openssl.conf > /dev/null 2>&1

执行上面会生成如下文件为
ls
admin-node1.csr      admin-node1.pem  ca.pem  member-node1.csr      member-node1.pem  node-node1-key.pem  openssl.conf
admin-node1-key.pem  ca-key.pem       ca.srl  member-node1-key.pem  node-node1.csr    node-node1.pem

查看新证书
查看CA证书有效时间
[root@node1 ssl]# openssl x509 -in ca.pem -noout -text | grep 'Not'
            Not Before: Jan 26 14:04:25 2021 GMT
            Not After : Jan 26 14:04:25 2031 GMT
            
查看CA签发的Etcd证书有效时间。
[root@node1 ssl]# openssl x509 -in member-node1.pem -noout -text | grep 'Not'
            Not Before: Jan 26 14:08:36 2021 GMT
            Not After : Jan 26 14:08:36 2022 GMT
[root@node1 ssl]# openssl x509 -in admin-node1.pem -noout -text | grep 'Not'
            Not Before: Jan 26 14:10:43 2021 GMT
            Not After : Jan 26 14:10:43 2022 GMT
[root@node1 ssl]# openssl x509 -in node-node1.pem -noout -text | grep 'Not'
            Not Before: Jan 26 14:12:44 2021 GMT
            Not After : Jan 26 14:12:44 2022 GMT
            
拷贝到其他etcd节点
scp -r /etc/ssl/etcd/ssl/ root@${other_node}:/etc/ssl/etcd/ssl/

重启etcd服务(记住，要3个节点一起重启，不然会hang住)
systemctl restart etcd

etcdctl --cacert=/etc/ssl/etcd/ssl/ca.pem --cert=/etc/ssl/etcd/ssl/node-node1.pem --key=/etc/ssl/etcd/ssl/node-node1-key.pem --endpoints=https://192.168.0.3:2379 endpoint health
https://192.168.0.3:2379 is healthy: successfully committed proposal: took = 16.529969ms

4 附加指令

.key 转换成 .pem：

openssl rsa -in temp.key -out temp.pem
.crt 转换成 .pem：

openssl x509 -in tmp.crt -out tmp.pem

原文始发于微信公众号（云原生内经）：Etcd证书过期解决方法

文章由极客之音整理，本文链接：https://www.bmabk.com/index.php/post/167984.html