Kubernetes Service: Scaling out after 24 hours from Cluster creation will fail because the Kubeadm Init Token TTL is only 24 hours.

[sangpengembara1979]

ISSUE TYPE

• Bug Report

COMPONENT NAME

Kubernetes Service

CLOUDSTACK VERSION

4.15.0.0

CONFIGURATION

plugins/integrations/kubernetes-service/src/main/resources/conf/k8s-master.yml
kubeadm init --token {{ k8s_master.cluster.token }} {{ k8s_master.cluster.initargs }}

OS / ENVIRONMENT

SUMMARY

Scaling out after 24 hours from Kubernetes Cluster creation will fail because the Kubeadm Init Token TTL is only 24 hours.

STEPS TO REPRODUCE

1. Create a new Kubernetes Cluster and wait after 24 hours when the Kubeadm Init Token is expired.
2. Scale out the Kubernetes Cluster and it will stuck in deploy-kube-system process because it will use the OLD Expired Token.
3. Going to the MASTER node, kubedm token list , there is no TOKEN since the INIT TOKEN is already expired.
4. Creating a new token on MASTER and modify the deploy-kube-system will make the new NODE able to join to cluster. Manually.

admin-kube-ha-master-1 core # kubeadm token list
TOKEN                     TTL       EXPIRES                USAGES                   DESCRIPTION                                                EXTRA GROUPS
5e595f.e0fde74405a4d346   6h        2021-03-02T09:04:28Z   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token

EXPECTED RESULTS

The kubernetes cluster should be able to scale out.

ACTUAL RESULTS

1. The kubernetes cluster will stuck at deploy-kube-system because it is using the old expired token.
2. After exceding the "cloud.kubernetes.cluster.scale.timeout" the cluster will be on ALERT state.
3. Going into the MASTER, kubeadm token list, there is no token, since the INIT TOKEN is already expired, and the new NODE still using the same token when doing deploy-kube-system.

[yadvr]

Fixed by #4747