HiveBrain v1.2.0
Get Started
← Back to all entries
snippetkubernetesMinor

How to Remove a Kubernetes (GKE) Cluster node-pool without down-time

Submitted by: @import:stackexchange-devops··
0
Viewed 0 times
withoutgkekubernetesnodetimeremovedownhowpoolcluster

Problem

I am using Google Kubernetes Engine.
I have a multiple Kubernetes cluster node-pools.
I want to delete one of them.

Is there a way that I can first migrate all the pods on one node-pool to another node-pool through the same kind of rolling update methods found in other features?

I thought it would do this automatically for me.
But when I deleted the node-pool I got downtime, luckily this was not a production environment.

I would like to know if there is a way to automatically migrate and move all pods from one node-pool to another node-pool and slowly roll the change when a node-pool is removed?

Thanks

Solution

If you cordon and drain the nodepool before deleting it, then you can avoid downtime. I use the following script (shamelessly taken from the lazyweb elsewhere and adapted to my needs):

oldpool=pool-1
oldnodes=$(k get no --selector='cloud.google.com/gke-nodepool='$oldpool -o json | jq .items[].metadata.name -r | xargs)

kubectl cordon --selector='cloud.google.com/gke-nodepool='$oldpool

for n in $oldnodes; do
  echo "draining $n"
  kubectl drain --delete-local-data --ignore-daemonsets $n
  echo "----------------------------"
done;

Code Snippets

oldpool=pool-1
oldnodes=$(k get no --selector='cloud.google.com/gke-nodepool='$oldpool -o json | jq .items[].metadata.name -r | xargs)

kubectl cordon --selector='cloud.google.com/gke-nodepool='$oldpool

for n in $oldnodes; do
  echo "draining $n"
  kubectl drain --delete-local-data --ignore-daemonsets $n
  echo "----------------------------"
done;

Context

StackExchange DevOps Q#4253, answer score: 3

Revisions (0)

No revisions yet.