How to Setup Master and Slave Architecture using Jenkins? (CI/CD:Part 2)?

This article will detail how to set up Master-Slave Architecture on Jenkins for high availability and scalability._

In the Master-Slave architecture of Jenkins, it is very common to have one master connected to several slaves.

Suppose a master node crashes due to an unexpected hardware failure. Your CI/CD service may be interrupted at some time.

In this post, you will learn

You can set up DNS services like Amazon Route 53 which can easily achieve failover architecture for Jenkins and ensure high availability.

It’s recommended to design the following Master-
Slave architect to integrate Github and Jenkins with Amazon Route 53.

You can refer below to understand Master-Slave architecture for details.

Flow:

Developer → GitHub → Amazon Route 53 → Master-Active and Master- Standby node → Multiple Slave nodes

When a developer pushes to Github, Github will send Push events to Master-Active or Master-Standby machine via Amazon Route 53.
Amazon Route 53 can determine available nodes to direct traffic to either master-active or master-standby.
If Master-Activeve dies, Route 53 can automatically switch traffic to the available node for Master-Standby.
To scale the capacity, you can add slave nodes behind the master node when needed via SSH connection. ( Refer to How to Setup Master and Slave Architecture using Jenkins? for details )
You need to set up the master-active to connect master-standby so that master nodes can failover each other to ensure high availability by the following settings in Pipeline.

agent {  
     node {  
          label ‘master-active||master-standby’  
     }  
}

With this setting, Jenkins’s Pipeline can select available nodes on Master- Active or Master-Standby to execute jobs.

This is tricky I would like to share some tips with you because this is not easy to find.

You may know master node in the Master-Slave architecture of Jenkins can assign a label to determine which slave node can run specific jobs.

However, how to smoothly dispatch multiple build evens to slave nodes?

The trick is you can assign multiple slave nodes with the same label in the pipeline.

agent {  
     node {  
          label ‘slave-group’  
     }  
}

Or you can assign different labels for different slave nodes.

agent {  
     node {  
          label ‘slave-Mac||slave-Win1||Slave-Win2’  
     }  
}

Is it possible to make the slave node run multiple jobs at the same time?

Yes, remember you need to uncheck below to allow your pipeline to run parallel jobs.

In this post, you have learned

how to make master nodes of Jenkins in high availability via Amazon Route 53.
how to scale capability by running current jobs on a single node or dispatch jobs to slave nodes by assigning the same label.

This can easily achieve horizontal scaling by adding new slave nodes with the same label in Jenkins’s pipeline for your CI/CD backend.

Hope this tutorial will be helpful for you.

If you found this article helpful, please follow us on Facebook to get the latest tutorials in the future.

Thank you for reading!