patternkubernetesMinor
kubernetes cluster node with 4cpu can't hold two pods with 2cpu
Viewed 0 times
canwithkubernetesnodepods4cpuholdtwo2cpucluster
Problem
I have a Kubernetes Cluster on GCE (Google Cloud Engine) (Also called GKE - Google Kubernetes Engine).
The Cluster is configured to have 4cpu per node. The pods I am using request 2cpu. I expect to be able to have 2 pods per node. But instead each time I launch a new pod, it also provisions a new node (am only getting 1 pod per node).
What is happening, How can I more efficiently use my resources? (I don't want to pay for nodes that are under-utilized)
Node:
Machine type
n1-highcpu-4 (4 vCPUs, 3.6 GB memory)
Pod (Yaml):
EDIT:
As mentioned below in the comments; it seems there are some 'default' pods that run on the node which utilize some small amount of CPU. For me there were two service pods taking up 100mcpu each. Additionally it seems there is a small overhead regardless of those small pods when viewing my node resources I see:
This means that each node in my cluster will only have 3.72 CPU that I can distribute to my pods. (3.92 allocatable, - 100m CPU per service pod (x2) = 3.72)
Knowing this, I can understand why my desired deployment will not work.
However, now this raises new questions; How can I most efficiently allocate pods that need 2cpu. Since GCE only lets me provision nodes with even numbers of cpu (2,4,6,etc) I will always have almost 2 wasted CPU. Is there a better solution?
EDIT2:
Settings my pods to use 1.8 CPU allowed me to deploy 2 pods per node. (and for now is good enough)
It would be desirable to be able to anticipate the overhead on nodes. For example, in my use-case above, allocating 4CPU per node, I should be able to know ahead of time that after the service pods and allocatable CPU are considered I will only be able to use 3.72 for my own pods. But if I now want to provision a new Node with 8CPU I will need to check these number
The Cluster is configured to have 4cpu per node. The pods I am using request 2cpu. I expect to be able to have 2 pods per node. But instead each time I launch a new pod, it also provisions a new node (am only getting 1 pod per node).
What is happening, How can I more efficiently use my resources? (I don't want to pay for nodes that are under-utilized)
Node:
Machine type
n1-highcpu-4 (4 vCPUs, 3.6 GB memory)
Pod (Yaml):
resources:
limits:
cpu: "2"
memory: 1000Mi
requests:
cpu: "2"
memory: 1000MiEDIT:
As mentioned below in the comments; it seems there are some 'default' pods that run on the node which utilize some small amount of CPU. For me there were two service pods taking up 100mcpu each. Additionally it seems there is a small overhead regardless of those small pods when viewing my node resources I see:
Resource type Capacity Allocatable
CPU 4 CPU 3.92 CPUThis means that each node in my cluster will only have 3.72 CPU that I can distribute to my pods. (3.92 allocatable, - 100m CPU per service pod (x2) = 3.72)
Knowing this, I can understand why my desired deployment will not work.
However, now this raises new questions; How can I most efficiently allocate pods that need 2cpu. Since GCE only lets me provision nodes with even numbers of cpu (2,4,6,etc) I will always have almost 2 wasted CPU. Is there a better solution?
EDIT2:
Settings my pods to use 1.8 CPU allowed me to deploy 2 pods per node. (and for now is good enough)
It would be desirable to be able to anticipate the overhead on nodes. For example, in my use-case above, allocating 4CPU per node, I should be able to know ahead of time that after the service pods and allocatable CPU are considered I will only be able to use 3.72 for my own pods. But if I now want to provision a new Node with 8CPU I will need to check these number
Solution
It seems like you found your answer but let me add a little color since we went through the same thing.
Your yaml for the pods can have both a request and limit for both CPU and memory. Since you are discussing CPU here I'll stick with that. What you can do in order to utilize your resources efficiently is set your resource request to the minimum requirement to get scheduled. Say 1.8. But also set the limit to a higher limit, say 2 or more. What this will do is make your pod be schedulable on the machine and if CPU is not completely consumed, your pod will utilize the available CPU up to 2 or the limit you set. This is the way kubernetes can make good use of the resources by "sharing" CPU to the application that needs it.
See here for requests and limits:
https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
Your yaml for the pods can have both a request and limit for both CPU and memory. Since you are discussing CPU here I'll stick with that. What you can do in order to utilize your resources efficiently is set your resource request to the minimum requirement to get scheduled. Say 1.8. But also set the limit to a higher limit, say 2 or more. What this will do is make your pod be schedulable on the machine and if CPU is not completely consumed, your pod will utilize the available CPU up to 2 or the limit you set. This is the way kubernetes can make good use of the resources by "sharing" CPU to the application that needs it.
See here for requests and limits:
https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
Context
StackExchange DevOps Q#4296, answer score: 1
Revisions (0)
No revisions yet.