snippetMinor
How can Agile methodologies, CI/CD be applied to massive, monolithic software projects with 100s/1000s of developers?
Viewed 0 times
canprojectsmassivemonolithicwith1000sapplied100sagilemethodologies
Problem
Using continuous integration represents an (almost) mandatory requirement for a software project to be agile. And without CI there can be no continuous delivery/deployment.
But CI can face scalability challenges, see How can continuous integration scale for very large projects/teams?.
Splitting work on multiple smaller pre-integration branches would allow work on those branches to be agile, but then merging of those pre-integration branches needs to be handled. And those would be massive, unpredictable merges, rather difficult to plan and incorporate into an Agile framework. Until the merge is actually performed one can't really tell if it'll take 2 hours, 2 days or 2 entire sprints to complete.
So how can Agile methodologies still be applied to massive, monolithic software projects with 100s/1000s of developers?
But CI can face scalability challenges, see How can continuous integration scale for very large projects/teams?.
Splitting work on multiple smaller pre-integration branches would allow work on those branches to be agile, but then merging of those pre-integration branches needs to be handled. And those would be massive, unpredictable merges, rather difficult to plan and incorporate into an Agile framework. Until the merge is actually performed one can't really tell if it'll take 2 hours, 2 days or 2 entire sprints to complete.
So how can Agile methodologies still be applied to massive, monolithic software projects with 100s/1000s of developers?
Solution
Well, you won't be able to apply an Agile methodology to a large team. One of usual principle is to work on Pizza Team (Less than 10 persons which can share a large pizza for diner together) because the lack of formalism advocated by Agile process makes it hardly applicable to a large team, the information would be scattered and some would be lost in the crowd.
So before thinking about the CI system, you'll have to review the organization around the software you build.
This mainly need to refactor a large project in smaller independent pieces, this could be separate softwares (and you'll take the path of microservices) or it could just be an architecture design within the software, where a class/object/library is given to a small team. This team has to document its service input and output (service contract/service orientation).
Once this is done you can split the responsibility of each independent piece to a smaller team which will be able to apply an Agile process.
This may solve the CI/CD problem, splitting the code base would allow each team to follow it's own pace, as long as it's interface is properly versioned and that the input/output are fixed for a type of call you can make parts of the product evolve while some other part keep a previous behavior.
The main drawback is that the service contracts from each part must be compatible between versions in use, which need a dependency system to know which part use which version of what before dropping an unused version of an API. This brings problems to make breaking changes and usually need a careful transverse planning between producer and consumers.
So before thinking about the CI system, you'll have to review the organization around the software you build.
This mainly need to refactor a large project in smaller independent pieces, this could be separate softwares (and you'll take the path of microservices) or it could just be an architecture design within the software, where a class/object/library is given to a small team. This team has to document its service input and output (service contract/service orientation).
Once this is done you can split the responsibility of each independent piece to a smaller team which will be able to apply an Agile process.
This may solve the CI/CD problem, splitting the code base would allow each team to follow it's own pace, as long as it's interface is properly versioned and that the input/output are fixed for a type of call you can make parts of the product evolve while some other part keep a previous behavior.
The main drawback is that the service contracts from each part must be compatible between versions in use, which need a dependency system to know which part use which version of what before dropping an unused version of an API. This brings problems to make breaking changes and usually need a careful transverse planning between producer and consumers.
Context
StackExchange DevOps Q#267, answer score: 4
Revisions (0)
No revisions yet.