I’ve covered a lot of topics about Allure and I hope all previous articles helped you to understand and implement this framework in your project. By the way, it’s growing actively and its creators are adding new features very often. For example, they released the 2.3 version with cool features a couple of days ago.
But.. you know… automated tests aren’t really useful if they aren’t run periodically, on time, sometimes even in a specific pipeline. That’s why we use continuous integration and delivery systems, and that’s why I decided to write one more article for finalizing the Allure series. As you understood, this post is about the integration of Allure framework into a CI system. I chose Jenkins for demonstration but all the rest have very similar workflows, so you can use whatever you like.
Challenge for this post:
To integrate Allure Reporting Framework to Jenkins continuous integrations system for the example demo project that we used in the previous articles.
Installing Allure Report plugin
Let’s start from the beginning. I assume that we have Jenkins installed, up and running already, so I will skip all these boring explanations. If you’re interested, here is the official guide how to install Jenkins. So, the very first step is to install Allure plugin to Jenkins:
- Go to Manage Jenkins -> Manage Plugins -> “Available” tab.
- Search for “Allure” and install it without restart.
This plugin adds the Allure post-build step (we will do it later) and it’s also responsible for displaying links to Allure reports and test history trend charts to the project’s page. The latest plugin version at the moment of this post creation was 2.20. But its creators aren’t sleeping, they are releasing new versions quite often, so check for updates.
Adding Allure CommandLine tool
We’ve installed the plugin but it doesn’t do magic by itself. We still depend on Allure CommandLine tool when we need to generate a report from JSON files. In order to add this tool we need:
- Go to Manage Jenkins -> Global Tool Configuration -> find Allure CommandLine block.
- Click on the “Add Allure CommandLine”, specify any name you like and then click on the “Add Installer”.
- I prefer to use “Extract *.zip” option. If so do you, then find the latest Allure CommandLine binary here and copy its direct link. Then just paste it to the Download URL text box.
If everything went well, you will have something like this:
That’s it! Just save the changes and let’s proceed with the project…
Setting up a new project
Have I mentioned in the past article that I was going to publish the example project on GitHub? Here it is! I am not gonna explain how to create projects in Jenkins in details – this topic is out of the scope of the Allure series of articles. What I want is to emphasize the most important key points for successful Allure integrations.
The first action is to connect the repository with the demo project to our Jenkins project. Nothing really complex, just don’t forget to add valid GitHub credentials in order to pull code successfully.
The second necessary action is to add a build step which actually builds our demo project and run the tests. As you remember from the past couple of articles, this exactly what the mvn clean test commands do. That’s why you should select “Invoke top-level Maven targets” option while adding a new build step. The Goals should be clean and test, of course. Also, don’t forget to expand the “Advanced” settings and specify a relative path to your pom.xml:
After we added a build step, we also need to add a post-build action which will generate a report. The appropriate drop-down list in the project configuration contains the “Allure Report” option that we’re looking for. The only one value that should be specified in this chapter is the path to the folder where JSON files are stored (AllureTestNGexamples\target\allure-results in our case):
That’s it! We can avoid using the allure serve commands – Allure Plugin for Jenkins takes care of everything, including publishing a report. We can run a new build now. No, it is even better to run several builds in a row with changing numbers of passed/failed tests because Allure has the nice feature called History Trend which displays historical data for the latest builds. Here is how it looks like on the project’s dashboard:
As you may notice, Allure Plugin added links to particular build reports (left bottom list of builds) and to the latest report (main project’s menu and in the middle of the screen). Moreover, if you click on the historical trend chart above the specific build number, Allure report for that specific build will be opened! Really nice, isn’t it?
Benefits of using Allure with CI system
Investing lots of efforts into improving Allure Framework yields results. For example, let’s open the latest generated report and check what extras we have there (see the picture gallery below for screenshots) in comparison to the local test run. And there we have:
- filled History Trend block in the main report tab;
- filled Executors block in the main report tab;
- additional History tab on each test page with info about 5 latest runs of a test.
The latest 2.3 version of Allure CommandLine tool also brought several improvements. For example, Suites tab in the main menu, or highlighted errors and stack-trace for failed tests, or embedded screenshots in the test window.
As you’ve seen already, Allure is a very powerful test report framework. It supports the most popular programming languages, continuous integrations systems, build tools and other frameworks. There is also the Plugin System which has some plugins and provides the API for creating your owns. And I have to mention that Allure is under active development, that’s important!
This is the final post from the Allure series. However, I don’t exclude additional posts in future, if something interesting is released by the team. Feel free to try Allure in your projects, explore the official documentation, ask me any questions here in comments.
Like this post?
Subscribe to updates from my blog, if you don't want to miss more interesting future posts and materials