kubernetes升级到v1.13.3版本

发表于： 2019-02-22 2019-02-24
分类： SRE
标签： Docker, https, I/O, k8s, kubernetes

升级133版本

kubernetes v1.13.3版本已经发布了，现在升级下：
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.13.md#other-notable-changes

检查新版本：

[root@node01 ~]# kubeadm upgrade plan
[preflight] Running pre-flight checks.
[upgrade] Making sure the cluster is healthy:
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: v1.13.0
[upgrade/versions] kubeadm version: v1.13.0
[upgrade/versions] Latest stable version: v1.13.3
[upgrade/versions] Latest version in the v1.13 series: v1.13.3

Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT   CURRENT       AVAILABLE
Kubelet     3 x v1.13.0   v1.13.3

Upgrade to the latest version in the v1.13 series:

COMPONENT            CURRENT   AVAILABLE
API Server           v1.13.0   v1.13.3
Controller Manager   v1.13.0   v1.13.3
Scheduler            v1.13.0   v1.13.3
Kube Proxy           v1.13.0   v1.13.3
CoreDNS              1.2.6     1.2.6
Etcd                 3.2.24    3.2.24

You can now apply the upgrade by executing the following command:

    kubeadm upgrade apply v1.13.3

Note: Before you can perform this upgrade, you have to update kubeadm to v1.13.3.

_____________________________________________________________________

[root@node01 ~]# kubeadm upgrade apply v1.13

[root@node01 ~]# kubeadm upgrade apply v1.13.3
[preflight] Running pre-flight checks.
[upgrade] Making sure the cluster is healthy:
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[upgrade/apply] Respecting the --cri-socket flag that is set with higher priority than the config file.
[upgrade/version] You have chosen to change the cluster version to "v1.13.3"
[upgrade/versions] Cluster version: v1.13.0
[upgrade/versions] kubeadm version: v1.13.0
[upgrade/version] FATAL: the --version argument is invalid due to these errors:

    - Specified version to upgrade to "v1.13.3" is higher than the kubeadm version "v1.13.0". Upgrade kubeadm first using the tool you used to install kubeadm

Can be bypassed if you pass the --force flag

先升级kubeadm

先用下代理，以免访问yum源超时：
export http_proxy='http://192.168.2.2:1080'
export https_proxy='http://192.168.2.2:1080'
export ftp_proxy='http://192.168.2.2:1080'
在各节点上都升级下kubelet kubeadm kubectl：

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF
# Set SELinux in permissive mode (effectively disabling it)
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
systemctl enable --now kubelet

1.13.3镜像

利用中转脚本,各节点都跑一遍：

#脚本来自https://it.520mwx.com/view/37277
#或者参考官方镜像https://github.com/openthings/kubernetes-tools/blob/master/kubeadm/2-images/
echo ""
echo "=========================================================="
echo "Pull Kubernetes v1.13.3 Images from aliyuncs.com ......"
echo "=========================================================="
echo ""

MY_REGISTRY=registry.cn-hangzhou.aliyuncs.com/openthings

## 拉取镜像
docker pull ${MY_REGISTRY}/k8s-gcr-io-kube-apiserver:v1.13.3
docker pull ${MY_REGISTRY}/k8s-gcr-io-kube-controller-manager:v1.13.3
docker pull ${MY_REGISTRY}/k8s-gcr-io-kube-scheduler:v1.13.3
docker pull ${MY_REGISTRY}/k8s-gcr-io-kube-proxy:v1.13.3
docker pull ${MY_REGISTRY}/k8s-gcr-io-etcd:3.2.24
docker pull ${MY_REGISTRY}/k8s-gcr-io-pause:3.1
docker pull ${MY_REGISTRY}/k8s-gcr-io-coredns:1.2.6


## 添加Tag
docker tag ${MY_REGISTRY}/k8s-gcr-io-kube-apiserver:v1.13.3 k8s.gcr.io/kube-apiserver:v1.13.3
docker tag ${MY_REGISTRY}/k8s-gcr-io-kube-scheduler:v1.13.3 k8s.gcr.io/kube-scheduler:v1.13.3
docker tag ${MY_REGISTRY}/k8s-gcr-io-kube-controller-manager:v1.13.3 k8s.gcr.io/kube-controller-manager:v1.13.3
docker tag ${MY_REGISTRY}/k8s-gcr-io-kube-proxy:v1.13.3 k8s.gcr.io/kube-proxy:v1.13.3
docker tag ${MY_REGISTRY}/k8s-gcr-io-etcd:3.2.24 k8s.gcr.io/etcd:3.2.24
docker tag ${MY_REGISTRY}/k8s-gcr-io-pause:3.1 k8s.gcr.io/pause:3.1
docker tag ${MY_REGISTRY}/k8s-gcr-io-coredns:1.2.6 k8s.gcr.io/coredns:1.2.6

echo ""
echo "=========================================================="
echo "Pull Kubernetes v1.13.3 Images FINISHED."
echo "into registry.cn-hangzhou.aliyuncs.com/openthings, "
echo "           by openthings@https://my.oschina.net/u/2306127."
echo "=========================================================="

echo ""

升级

kubeadm upgrade apply v1.13.3
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.13.3". Enjoy!
检查下：

[root@node01 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:08:12Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:00:57Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

升级之后

虚拟化cpu占用从50%下降到28%左右：

apiserver的cpu占用问题已解决，目前不超过20%，以前125%左右
etcd心跳丢失的问题已解决
namespace一直处于删除状态的问题已解决
iostat的kB_wrtn/s从以前600+变成现在的270不到
etcd执行超时和server过载问题没解决，可能是磁盘的确太渣了？

joker

174