Jenkins is an open source continuous integration server to automate build, test and deploy pipelines and even more. Jenkins pipelines and libraries have a big benefit as they can be added to SCM. However, testing of pipelines and libraries can be a hassle sometimes, especially when not dealing with it on a daily basis. One way to ease the pain of testing changes or new configurations is to know the different ways how these can be tested.
The “Replay” feature allows for quick modifications and execution of an existing Jenkinsfile without changing the pipeline configuration or creating a new commit. But remember that replay changes are only temporary, so don’t forget to either update the configuration or commit to SCM once you are satisfied.
Depending on how Jenkins libraries are configured within the project they may also appear in the replay overview. But you won’t see the library part if they are configured as global libraries compared to folder libraries.
Often when creating a new Jenkinsfile it is often easier to start developing it directly within the Jenkins configuration. This way changes and adjustments can be tested within the Jenkins job without committing and pushing every single change.
To create a new pipeline job for testing can be an option to not interfere with the main builds and to safely test it. If multiple jobs are involved you may need to create copies of multiple jobs if their configuration changed.
One benefit of using SCM for Jenkinsfiles is the possibility of creating branches for any needed changes. I can only recommend it as any changes can go through a review process which is very beneficial.
For Jenkins libraries branching is often the only option to test changes without affecting the stable master branch. Thus it comes in handy that library branches can be added to a Jenkinsfile and not only one but multiple libraries can be tested together at the same time.
Jenkins shared library is a powerful way for sharing Groovy code between multiple Jenkins pipelines. However, when many Jenkins pipelines, including mission-critical deployment pipelines, depend on such shared libraries, automated testing becomes necessary to prevent regressions whenever new changes are introduced into shared libraries. There is no built-in support for automated testing, and the Jenkins Handbook even suggests doing manual testing via the Jenkins web interface. There is a better way.
Before you can start writing tests, you need to add a few things to your Jenkins library project:
A JUnit library, Jenkins Pipeline Unit, for simulating key parts of the Jenkins pipeline engine
A build tool, for compiling and running tests
A build script, for specifying dependencies and other project configuration
As the build tool I’d recommend Gradle, because it’s fast, flexible, and very Groovy oriented.
brew install gradle
After installing Gradle, add the Gradle wrapper to the project directory.
cd <project dir>
To run the tests, run the Gradle test task in the project directory, potentially with artifactory login.
cd <project dir>
To run a single test class, or even a single test, run Gradle with the –tests option:
./gradlew test –tests MyClassTest
./gradlew test –tests MyClass.checkConfig
Finally to add the build script and jenkins-pipeline-unit to the project, we need to created a file named build.gradle.
apply plugin: ‘groovy’
sourceCompatibility = 1.8
// all code files will be in either of the folders
This will result in a report as follows which lists the executed tests and their status.
Even though all information is present it is hard to visually differentiate between the failures and successes and maybe a summary would be nice too. Thus, to make the report even more useful and readable we can instead use the following script part in the build.gradle script.
Tests that use Jenkins Pipeline Unit are written as JUnit test classes that extend com.lesfurets.jekins.unit.BasePipelineTest. This class provides properties and methods for accessing the script variable binding, loading scripts, creating mock steps, and simulating other features of Jenkins pipelines.
To make things easier and reduce code duplication within each test I would advice to create your own TestBase which can take care of the test setup which is the same for all tests.
Testing Jenkinsfiles and libraries can be tedious so knowing how to best test them is a huge advantage. Each change should ideally be tested manually at first to make sure it actually does what is expected in the actual environment. To avoid regressions issues and side effects of changes it is advised to add automated tests to a library. This will not only minimize the trouble but also increase the confidence when dealing with Jenkins scripts. Happy testing!