Running Jenkins job simultaneously on all nodes

Question 1

Rundeck might be a tool better suited to your needs. Can be setup to run several jobs in parallel and has a plugin for Jenkins: http://rundeck.org/
Rundeck is designed to integrate with larger systems. We generate the resource file from our configuration management database. Very easy to do see the documentation: http://rundeck.org/docs/administration/node-resource-sources.html.
Additionally plugins available for amazon and/or systems like puppet and chef: http://rundeck.org/plugins

Question 2

Install
- Parameterized Trigger Plugin
- NodeLabel Parameter Plugin
For the job you want to run, enable Execute concurrent builds if necessary
Create another job besides the job you want to run on all slaves and configure it
- Build > Add build step > Trigger/call builds on other projects
  - Add ParameterFactories > All Nodes for Label Factory > Label: the label of the nodes

Question 3

The matrix build will work; use "Slaves" as the axis and expand the "Individual nodes" list to select all of your nodes.

Note that you will need to update the selection every time you add or remove a slave.

For a more maintainable solution, you could use the Job DSL plugin to set up a seed job that has the template for the build, then loops over each slave and creates a new job with the build label set to the name of the slave.

Question 4

There is two plugins that you need: Paramitrized Trigger Plugin to be able to trigger other jobs as build step of your main job, and NodeLabel Plugin (read the BuildParameterFactory section for descrition of what you need) to specify the label.

Question 5

The best and easiest way to accomplish this is using Elastic Axis plugin.
1. Install the pulgin.
2. Create a Multi Configuration job.(Install if not present)
3. In the job configuration you can find new axis added as Elastic axis. Add the label as shown below to get the job run on multiple slaves. enter image description here

Question 6

Taking a few of the above answers and adjusting them for 2.0 series.

You can now launch all a job on all nodes.

// The script triggers PayloadJob on every node.
// It uses Node and Label Parameter plugin to pass the job name to the payload job.
// The code will require approval of several Jenkins classes in the Script Security mode
def branches = [:]
def names = nodeNames()
for (int i=0; i<names.size(); ++i) {
  def nodeName = names[i];
  // Into each branch we put the pipeline code we want to execute
  branches["node_" + nodeName] = {
    node(nodeName) {
      echo "Triggering on " + nodeName
      build job: 'PayloadJob', parameters: [
              new org.jvnet.jenkins.plugins.nodelabelparameter.NodeParameterValue
                  ("TARGET_NODE", "description", nodeName)
          ]
    }
  }
}

// Now we trigger all branches
parallel branches

// This method collects a list of Node names from the current Jenkins instance
@NonCPS
def nodeNames() {
  return jenkins.model.Jenkins.instance.nodes.collect { node -> node.name }
}

Taken from the code https://jenkins.io/doc/pipeline/examples/#trigger-job-on-all-nodes

Question 7

Got it - No need for any special plugin!

I've created a parent job that triggers/call another build , And when I'm calling him I pass him the Label that I wan't the child job to run on.

So basically the parent job Only triggers the job I need , and the child job will run as many times as the number of slaves in that Label (In my case 4 times).

enter image description here

Question 8

I was looking for a way to run docker system prune on all nodes (with label docker). I ended with a pretty simple scripted pipeline, which AFAIK will only need the pipeline plugin to work:

#!/usr/bin/env groovy

def nodes = [:]

nodesByLabel('docker').each {
  nodes[it] = { ->
    node(it) {
      stage("docker-prune@${it}") {
        sh('docker system prune -af --filter "until=1440h"')
      }
    }
  }
}

parallel nodes

What this does, it is looking for all nodes with label docker, then iterates over it and creates an associative array nodes with one step per found node (to be precise, what this is doing is cleaning all old docker stuff older then 60 days). parallel nodes starts to execute in parallel (on all found nodes simultaneously).

Hope that this will help someone.

Question 9

Enable This project is parameterized, add a parameter of type Label, enter an arbitrary name for the label and select a default value such as a label covering a number of nodes or a conjuction (&&) of such labels. Enable Run on all nodes matching the label, keep Run regardless of result, keep Node eligibility at All nodes.

Question 10

Solution: You can succinctly parallel the same build across multiple Jenkins nodes

This can be useful for building the same project on different environments ( for example: build node applications on test ,dev and staging environments )

Example:

pipeline {
agent { docker {  image 'node:14-alpine' } }
stages {
    stage('build') {
        steps {
            parallelTasks
        }
      }
   }
}


def parallelTasks() {
  def labels = ['test', 'dev', 'staging'] // labels for Jenkins node types we will build on
  def builders = [:]
  for (x in labels) {
      def label = x
      
      builders[label] = {
          node(label) {
              sh """#!/bin/bash -le
                    echo "build app on ${label} node"
                    cd /home/app
                    npm run build
              """
          }
      }
  }
  parallel builders


}