patterndockerMinor
Cheapest and easiest way to run a periodic Docker process
Viewed 0 times
easiestprocessdockercheapestwayperiodicandrun
Problem
I have a dockerized Python process that a client would like to run once or twice a month. It calls a number of Google APIs and stores the results in a Google Sheet. It does not accept network requests. I want to provide one "magic button" that will spin up the Docker container and then shut everything down.
Some background: I had previously deployed this to an AWS EC2 micro instance via Docker Cloud. My client got a Docker Cloud account and was able to run the container on demand, relatively painlessly. There are two problems with this workflow:
the Docker container was run up or not. This became expensive.
Alternatives seem to be Google and AWS Kubernetes services. My reservation is that they will be too complicated for my client to use. Advice?
Some background: I had previously deployed this to an AWS EC2 micro instance via Docker Cloud. My client got a Docker Cloud account and was able to run the container on demand, relatively painlessly. There are two problems with this workflow:
- The provisioned EC2 instance (Docker host) was running 24/7, whether
the Docker container was run up or not. This became expensive.
- Docker cloud shut down a month ago.
Alternatives seem to be Google and AWS Kubernetes services. My reservation is that they will be too complicated for my client to use. Advice?
Solution
AWS offers fargate now, which is managed by them rather than being a service like ECS which you ran the servers for and they manage the control plane. Fargate is basically "serverless" ECS. You can use a cloudwatch scheduled even to run something occasionally on a schedule on a fargate ECS cluster. Therefore you're only paying for what you're using. Something to consider is fargate is quite a bit more expensive than ECS running on your own EC2 instance, but if you're running in frequently you'll likely see a pretty marginal savings.
Another option would be to automate something similar. So create an ECS cluster with an ASG backing it, and adjust scale to match ECS demand, and then use cloudwatch to launch the container. ECS can then scale up and down, but this can be tricky because this method is usually used to autoscale ECS clusters based on cloutwatch triggers, so you might have to hack around in cloudwatch and possibly hack together a lambda script, since you're technically not using it the way it's intended. In the end this would probably be the most cost effective, but in your case the cost savings could be negligible.
In my opinion Kubernetes is a ten ton hammer that you'd be using to hang a picture on the wall. It's way overkill.
You can explore those 2 options and see if they are cost effective, and easy enough to manage. Other container orchestration is usually going to be overkill, ECS has the advantage(usually a disadvantage) of being stupid simple and you pay very little and have basically no operational overhead for the control plane.
Another option would be to automate something similar. So create an ECS cluster with an ASG backing it, and adjust scale to match ECS demand, and then use cloudwatch to launch the container. ECS can then scale up and down, but this can be tricky because this method is usually used to autoscale ECS clusters based on cloutwatch triggers, so you might have to hack around in cloudwatch and possibly hack together a lambda script, since you're technically not using it the way it's intended. In the end this would probably be the most cost effective, but in your case the cost savings could be negligible.
In my opinion Kubernetes is a ten ton hammer that you'd be using to hang a picture on the wall. It's way overkill.
You can explore those 2 options and see if they are cost effective, and easy enough to manage. Other container orchestration is usually going to be overkill, ECS has the advantage(usually a disadvantage) of being stupid simple and you pay very little and have basically no operational overhead for the control plane.
Context
StackExchange DevOps Q#4341, answer score: 3
Revisions (0)
No revisions yet.