snippetMinor
How to handle large deployments using tools like gitlab and docker
Viewed 0 times
handledockergitlablikelargedeploymentsusinghowandtools
Problem
How to deploy multiple instances (40+) of the same application with gitlab? (it is a simple tomcat web app)
I want to be able to individually upgrade/downgrade them using web interface.
We already have a working gitlab+docker pipeline that deploys one instance of an app. We are not using k8s yet.
Should we create gitlab environment for each instance? Do we have better options besides gitlab environments?
Update:
Instances (websites) do differ, they have the same codebase but different configuration. Right now the scaling is not a priority, what we really want is to have an interface to add, deploy, upgrade and downgrade websites. I've tried gitlab environmets on a smaller project and it's really awesome when you have like one or two websites, but when you have 10 or more and you have one environment for each website .gitlab-ci.yml grows exponentially.
I want to be able to individually upgrade/downgrade them using web interface.
We already have a working gitlab+docker pipeline that deploys one instance of an app. We are not using k8s yet.
Should we create gitlab environment for each instance? Do we have better options besides gitlab environments?
Update:
Instances (websites) do differ, they have the same codebase but different configuration. Right now the scaling is not a priority, what we really want is to have an interface to add, deploy, upgrade and downgrade websites. I've tried gitlab environmets on a smaller project and it's really awesome when you have like one or two websites, but when you have 10 or more and you have one environment for each website .gitlab-ci.yml grows exponentially.
Solution
The documentation shows that you can have 40+ branches with different configurations. You express concern that the configuration becomes large. With a good editor that isn't really a problem. Yet that feels wrong, right?
With git, in general, a repository per independent deployable works better than a single repository holding many independent deployables. So an alternative layout is not 1 repo with 40 branches, but 1+40 repos. That would be 1 repo that builds the latest code and creates releases. Then 40 repos that represent the configuration to deploy a particular release of the code configured for one website. The 40 repos that deploy the code can hold the 40 per-website configurations and a standardised build script that simply downloads the right release of the code to deploy along with the website configuration into the environment.
We use 1+N+M git repos. We have 1 repo that holds the deploy scripts. They are versioned and released. We have N repos that build the different apps into releases. Currently, two php apps and two react apps that work together but are deployed and upgraded independently. Then M repos that represent the environments/sites we deploy into which is currently only staging and live.
We are using k8s (OKD) and the deployment scripts are using helm. That just means that our deployment scripts and environment setup are simpler and rollback/downgrade is fully automated. We don't actually download the app to deploy during our build. Instead, we simply push out the docker registry tag of the version of the code we want to deploy into a given environment.
I started a project to opensource our approach to deploying apps from git which is called OCD. It is still a work in progress but it demonstrates some of the above ideas.
With git, in general, a repository per independent deployable works better than a single repository holding many independent deployables. So an alternative layout is not 1 repo with 40 branches, but 1+40 repos. That would be 1 repo that builds the latest code and creates releases. Then 40 repos that represent the configuration to deploy a particular release of the code configured for one website. The 40 repos that deploy the code can hold the 40 per-website configurations and a standardised build script that simply downloads the right release of the code to deploy along with the website configuration into the environment.
We use 1+N+M git repos. We have 1 repo that holds the deploy scripts. They are versioned and released. We have N repos that build the different apps into releases. Currently, two php apps and two react apps that work together but are deployed and upgraded independently. Then M repos that represent the environments/sites we deploy into which is currently only staging and live.
We are using k8s (OKD) and the deployment scripts are using helm. That just means that our deployment scripts and environment setup are simpler and rollback/downgrade is fully automated. We don't actually download the app to deploy during our build. Instead, we simply push out the docker registry tag of the version of the code we want to deploy into a given environment.
I started a project to opensource our approach to deploying apps from git which is called OCD. It is still a work in progress but it demonstrates some of the above ideas.
Context
StackExchange DevOps Q#5550, answer score: 3
Revisions (0)
No revisions yet.