HiveBrain v1.2.0
Get Started
← Back to all entries
patternkubernetesMinor

Meaning of metrics in Google Cloud Console

Submitted by: @import:stackexchange-devops··
0
Viewed 0 times
meaninggooglecloudmetricsconsole

Problem

I have a 3-node kubernetes cluster with 3 n1-standard-1 :1 vCPU, 3,75 Go RAM, in a gcloud projet. Recently, I ran a test that created quite a few jobs (about 150). Half the jobs were running an hello world script, the other half were failing due to configuration error, and those jobs kept retrying by restarting new pods.

In my console I observed the following graph:

It seems that my cluster is capped at 3% of maximum utilization, if I read the units correctly. What bothers me is that in my test, the amount of pods/utilisation should not be constant, so I was not expecting the utilization to be a flat line like this.

Also, I noticed about about 30 pods were in unschedulable state, so my initial thought was that I reached maximum utilization of the instance.

EDIT

It is clear now that my test is too poorly designed, so I cannot conclude about maximum utilization so far.

Solution

It seems that my cluster is capped at 3% of maximum utilization, if I
read the units correctly. This makes no sense at all, why isn't it
capable of reaching 100% ?

Why do you think it does make no sense at all?


I ran a few tests that created quite a few pods

What is a few tests?


(each pod running basically an hello world script)

What does this script do? Print helloworld? If that is the case then it could be possible that the cluster will not use 100%.

By the way, how large is the cluster? How many agents? How many memory? So if the cluster possesses 1024GB of memory then it could be the case it is using 3%.

In summary, to indicate whether something makes no sense at all a number of facts is required:

  • content of the helloworld script



  • characteristics of the cluster, e.g. totol cpu, memory, type of disks, storage capacity, number of agents, masters



  • definition of few tests and few pods. How many tests were run, frequency, e.g. 10/s, how many pods



If you would really like to loadtest a k8s cluster, kubernetes has some documentation about this, e.g. this document explains how to load test a k8s cluster and in this case force autoscaling.

Context

StackExchange DevOps Q#2960, answer score: 4

Revisions (0)

No revisions yet.