How to properly scale Jenkins?

In my project we have one AWS server running Jenkins Master + 1 Jenkins slave (2 executors)... and we need more

In order to augment our build power we have three options:

• Scale up: Make AWS instance bigger and add more executors.

• Scale up: Make AWS instance bigger and add another jenkins slave process.

• Scale out: Create another AWS instance with a jenkins slave and connect it to master

We want to do 2. as we are in a big organization and our current Jenkins Master has already access to every place he needs. Option 3. "New server" is complicated as it needs more bureaucratic approvals that will take weeks.

So my questions are:

• Is there any technical issues in option 2?. Maybe the executors of each jenkins slave are not aware of the other slave executors?

• In general, what is the best approach to scale Jenkins? Scaling up or scaling out?

There are no fundamental technical issues with running multiple jenkins slaves on the same machine. In fact Running Multiple Slaves on the Same Machine lists several good reasons for doing it:


While the correct use of executors largely obviates the need for
multiple slave instances on the same machine, there are some unique
use cases to consider:

• You want more configurability between the configured nodes. Say you have one node set to be used as much as possible, and the other node

to be used only when needed.

• You may have multiple Jenkins master installations building different things, and so this configuration would allow you to have

slaves for more than one master on the same box. That's right, with
Jenkins you really can serve two masters.

• You may wish to leverage the easiness of starting/stopping/replacing virtual machines, perhaps in conjunction with Jenkins plugins such as

the Libvirt Slaves Plugin.

• You wish to maximize your hardware investment and utilization, at the same time minimizing operating cost (e.g. utility expenses for

running idling slaves).


In general scaling out is preferred, primarily because ability to scale up is typically limited by the types/sizes of the available physical resources.

In particular for augmenting build power I'd recommend an analysis of your actual build to determine how it uses the machine resources, which/where its bottlenecks are and what scalability limitations it raises to reveal if scaling up even helps.

For example I encountered cases in which the build time for 2 parallel builds on the same machine was longer than the combined build times of the same 2 builds executed sequentially (non-overlapping) on the same machine. In such case I wouldn't even consider scaling up as it would actually decrease the overall building capacity.

StackExchange DevOps Q#449, answer score: 14