Don’t let go of the Handlebars…

Creating a new HTML reporter for Newman has been a wonderful learning experience – I’ve managed to bring together several different skills, some that I had knowledge of already and many that I actually taught myself,  by failing lots, as I moved through the development process.

The biggest thing that I took away from all this was the fact that I really suck at designing and styling reports. Personally, I don’t think it’s that bad and it could have been a lot worse.

 

The new reporter needed to have a ‘default’ template, this was to ensure that it can work out of the box once it’s either been installed globally or part of a project. With it being the default, it’s not going to be to everyone’s taste – This might not be just to do with the style or the navigation, it might also be the level of detail on the report.

I’ve had numerous bit’s of awesome feedback about the report and suggestions for different things that could be improved. I’ve made many changes off the back of this feedback but I’ve stopped short of changing everything suggested.

The beauty of the HTML reporters is that you can supply your own custom templates and create more meaningful reports that suit your own context, the underlying data from the Newman summary object, is the same for each. The way that you choose to display this is open to your own personal taste and requirements.

I often reply to people with something worded similar to the paragraph above, I love that people want to download and use something that I’ve created, it makes me feel awesome but I’m not forcing anyone to just use the template that I’ve created.

This got me thinking though, creating a template to me is easy-ish but I’m suggesting to people that are not me, to just go and create one. This is not actually very helpful of me and I love to help people…

I wanted to just explain the basics of getting data from Newman and render this on a HTML page – This is not going to be in-depth and you’re not going to be a template creating ninja as a result but at least you can increase your own understanding about how it all pins together.

This reporter and all the other Newman HTML reporters that I have seen, uses a template engine called Handlebars  – There are many template engines for JavaScript that do a similar thing but this is the one currently used in the reporters.

How do we get the Newman data onto a HTML page?

First off we need to create a new file, in your favourite text editor, with a `​.hbs` file extension. This will be the filename that we will be referencing, when we run Newman from the command line or within a script.

The file can contain something like this for now, this is just for demo purposes to add some structure to the page.

The Newman Summary is a JSON object which contains all the details relating to the collection run. Using the Handlebars syntax we are able to reference different parts of the summary object in our template page and render these values, from the data, on our final report.

The Handlebars library contains lots of different options but I’m only going to run through a couple to explain what’s happening with the data – Once you’re happy, you can go and explore all the other options open to you.

I’ve taken a sample of the data, the whole object can be quite verbose and displaying the whole thing will be confusing so this sample set will be fine for now. I’ll add a link to a full sample, at the bottom of the post.

Above, we have the stats and the timings data for the test run, we’re going to use the {{#}} and {{/}} block expressions on our template to help us render those values on the report. The whole object is the summary, so using the with helper, we create a {{#with summary}} block to tell Handlebars that we want to use that part of the data.

Each time you open a block expression, you need to remember to close it out again using the name of the helper, on this occasion it’s {{/with}}. So now we have the summary block, we need do the same again to reference the data within the stats object.

From here, we can start to directly reference parts of the data – I want to render the ‘total’ and ‘failed’ values from the iterations, requests, prerequestScripts and testScripts. Each name is the same as the key in the sample data set above.

Let’s run Newman with our basic template…

We are now in a position to run the a Postman Collection using Newman, we’ll be running this from the terminal but we could also create a nodejs script, to do the same thing.

Ensure that you have both the newman and newman-reporter-htmlextra npm modules globally installed.

npm install – g newman newman-reporter-htmlextra

To run the collection, you can use the following command:

newman run -r htmlextra –reporter-htmlextra-export ./report.html –reporter-htmlextra-template ./handlebars.hbs

By using the –reporter-htmlextra-template flag and giving it the path to the template file, the reporter will now use this to create the report. By default, if you do not supply an export location, it will create a ./newman directory for you and add all the newly created reports in here. Specifying the export location and the report name, will give you a far better level of control.

Once the test run has completed, It will create a very unimpressive looking report – My default template is not looking too bad now… 🙂

Obviously, that’s not going to win any style awards but it’s just giving you a feel for how the data will be rendered on a browser window, from the reference points in the template.

As the template is basically a static HTML page, we can add the standard HTML elements/tags to it and add a tiny level of formatting to the report.

For this example, I’ve created a basic`table` and added some CSS to place a border around it.

There are other standard Handlebar helpers available for us to use that might be familiar to you, if you’ve coded in any programming language – The each helper will give us the ability to loop through an array and render each value on the report.

This is very useful for extracting the data from each of the individual requests in the collections or like I’m doing here, each of the failures.

There are a few of things happening here and I wanted to provide a short example of using different helpers as well as adding some logic into the mix, that will display certain things depending on the data set.

I’ve `wrapped` everything in an {{#if}} {{/if}} block, by doing this I’m controlling the data that’s rendered on the report.

If we don’t have any failures during the test run, we don’t really want to display an empty set of headings on the final report – I’ve achieved this by checking the failures array using the summary.failures.length condition. If the array contains any failures it will use that data to ​`​fill` all the reference points.

Coupled with this is the {{#each}} {{/each}} block, this expression will just loop through the array and grab the data required from each of the objects. This could iterate/loop over 1 object or a 1000+ objects.

The Newman test run for this example, had 2 failures in the array, so we looped through those objects and extracted the values from the reference points, that we specified in the template code above.

This resulted in the report that you can see below. Still no very ‘pretty’ but you can see the data coming through and it’s the start of making a beautiful report 🙂

Lastly, something I didn’t mention was the use of the {{#gt}} {{else}} {{/gt}} block expression. The `gt` condition means ‘greater than’, I use this quite a lot to change certain elements on the report.

In this case, I’ve used it to change the text in the title to help pluralise the sentence. It frustrates me when I see websites that have labels like ‘1 failures’ or ‘3 failure(s)’ – By controlling which text is used based on the number in the {{#gt}} condition, I don’t have to compromise and display the incorrect label.

Making things look less boring…

There are lots of frameworks out there that can help you create a styled web page relatively quickly, my ‘go to’ has always been Bootstrap and it’s been awesome. The framework comes packed full of classes that, once the framework is referenced in the file, allow you to provide different styles to all the HTML elements on the report.

I’ve added the code block here so that, If you wanted too, you could paste this into your .hbs file and create the same thing locally with the newman-reporter-htmlextra module.

This template example is quite basic and is using some of the bootstrap classes to change things like the header text to uppercase, the text alignment, the ‘danger’ colour in the table cell for the failed assertions etc.

You may have seen this line {{totalTests stats.assertions.total skippedTests.length}} in the template. When I created the new reporter, I wanted to add in some additional helpers to the Handlerbars engine, you have the ability to register helpers that will assist you to display the data in a certain way.

I added the totalTests handler so that the total number of assertions takes into account the number of skipped tests, that there may have been during the run. The helper does a simple calculation on the values passed in to it and renders the correct data on the report.

Adding lots of different bespoke handlers is time consuming, luckily for me, someone has taken that pain away and has created an extensive list of handlebar helpers, these have all been included in the new reporter so you can use anything in that module, to help you create some beautiful looking and more meaningful reports, with far little effort.

Bootstrap is not the only framework out there – Bulma is also something that can be used in the template. This is well worth checking out, I plan on creating more report templates using this framework and give the users a better selection of ones to use in the reporter and not just be stuck using the default one.

---

This is an example of just a standard Newman Summary object that is similar to the one that would be used to extract different pieces of the data and render these on a report.