How to define dynamic parallel stages in a Jenkinsfile?

In declarative pipelines, Jenkins allows the definition of parallel stages. It further allows scripted pipeline general purpose scripts to create and manipulate the artifacts of the declarative pipeline. For example, it can create stages dynamically.

I would like to know how I can dynamically create parallel executing stages. Please note I am not talking about steps or actions, which are solved here:

How to properly achieve dynamic parallel action with a declarative pipeline?

Below is a simple Jenkinsfile that dynamically adds stages using groovy code. These stages are sequential. I would like a Jenkinsfile that adds stages dynamically as below, but uses the parallel construct at the stage level. As a result, three stages that run parallel should be generated by the program.

pipeline {
agent any
stages {
stage('Add regression tests') {
steps {
addStage('offline','open-source','webgoat')
addStage('offline','open-source','juice-shop')
addStage('online','internal','our-website')
}
}
}
}

final void addStage(final String type, final String suite,final String application) {
script {
stage("Run ${type} application '\n${application}' in test suite '${suite}'") {

runTestOnApplication(type,suite,application)

}
}
}

final void runTestOnApplication(final String type, final String suite,final String application) {
labelledShell label: "Run test shell script for ${type} application '${application}' in test suite '${suite}'", script: 'test.bash'
}



This is quite complicated to achieve in Jenkins. We had a similar issue and here's how we solved it:

• We have a shared library where we keep our scripts ins ./vars/someScriptName.groovy

• We created a groovy script that generates the stages.

Here's a working pipeline code for you to try it out:

def generateITParallelStages(body)
{
    def config = [:]
    config.stages = []

    body.resolveStrategy = Closure.DELEGATE_FIRST
    body.delegate = config
    body()

    def buildStages = [:]
    config.stages.each { String stage, closure ->
        // default settings, if you want to have any
        def settings = [:]
        settings.type = 'online'; // if you don't define the type, it will be 'online' by default
        settings.suite = 'internal'; // if you don't define the suite it will be 'internal' by default

        closure.resolveStrategy = Closure.DELEGATE_FIRST
        closure.delegate = settings
        closure()

        buildStages.put(stage, prepareStage(stage, settings))
    }

    return buildStages
}

def prepareStage(String stage, Map settings)
{
    return {
        print "Run ${settings.type} application  '\\n${stage}' in test suite '${settings.suite}'"
    }
}

pipeline {
    agent any
    stages {
        stage('Add regression tests') {
            steps {
                script {
                    parallel generateITParallelStages {
                        stages = [
                                webgoat: {
                                    type = 'offline'
                                    suite = 'open-source'
                                },
                                'juice-shop': {
                                    type = 'offline'
                                    suite = 'open-source'
                                },
                                'our-website': {
                                    type = 'online'
                                    suite = 'internal'
                                }
                        ]
                    }
                }
            }
        }
    }
}

And here's the result:

This is the only way we have been able to dynamically generate the parallel stages. Everything else will fail (at least at the time of writing this).

Other people seem to be having a hard time figuring this out as well - JENKINS-53032. If you feel the same make some noise :)

StackExchange DevOps Q#9887, answer score: 3