Tool preferences for rapidly spinning up and spinning down k8s clusters

My team hosts several completely different projects on a production k8s cluster with a given configuration. We use AWS (and are starting to use Google Cloud). We have a QA cluster as well, that "sort of" mimics the production configuration, and has a few test projects deployed on it as well. I say "sort of" because right now we just have a collection of shell scripts and custom commands that an engineer manually ran to create the clusters. This had a bad consequence when we made a change to the configuration on the QA and production cluster, and it prevented deployment to that cluster until we reverted the change.

I am looking to implement Infrastructure as Code (IAC) and a corresponding CI/CD pipeline to prevent this. The goal is that a PR will trigger a build that will spin up a cluster by following the IAC steps, deploy some projects on it, run some tests, then spin down the cluster and provide a test report.

I am investigating Kops, Ansible, Chef, and Puppet as candidates for this task. Right now the shell scripts I mentioned before are mostly kops commands. I have been spending the last few days reading about the other three technologies as well, and I am not quite sure how they might fit into what I need.

Are there any recommendations for my particular use case? Implement IAC and a CI/CD pipeline that can spin up a new k8s cluster with the proposed changes and then spin it down once tests finish?

You could try this:

https://github.com/heptio/aws-quickstart

Their CTO uses it to spin up K8S clusters for his weekly youtube live stream called TGIK, in order to showcase whatever he wants to talk about.

I myself use GKE so I do not need it.

StackExchange DevOps Q#3961, answer score: 2