Hipchat Custom Commands – Continually Improving our processes!

At NewVoiceMedia, like many companies, we hold Hackathons each quarter. This is a chance for everyone (not just developers) to create something that will be of benefit to our customers or help improve an internal process of the business. These small projects could be a proof of concept or a small feature that is ready to go straight into the Live product.

In the past, I’ve helped out on tiny projects with a couple of developers but for the last few hackathons, I’ve used it as a chance to improve my coding skills and create tools to help out other people. There has been a process that has bugged me for a while and we do the task every week, so I thought that it was a perfect place for improvement. We manually create/construct a URL that displays the release notes for our service, based on the “start” and “end” commit numbers, the “start” number being our deployed production version.  Like I said, this is a weekly task as that’s our current release frequency. I knew there was a better way and more efficient way to get this done. The URL in the images below is an internally used NVM address so it, won’t work anywhere outside of our Company.

A while back I read a great post by Neil Studd, He created a bot that was accessed via a custom slash command within the Slack platform. I thought it was impressive and it was something that I wanted to try to replicate myself! So I had a slight problem at the very beginning, we don’t use Slack at NVM (not my choice), we use Hipchat instead. I needed to figure out how I could get a custom slash command to work with that platform, thankfully, they do have their own version of this feature.

Similar to Neil’s method, I wanted to use Heroku to host my application, I mainly wanted to try this out because it was unfamiliar to me and it was a great opportunity to learn something new!!

Creating an MVP

I’ve recently been dabbling in NodeJS and creating little test tools that I use on my team, we also have a production API that uses this language so I’m quite comfortable in that space at the moment. For what I wanted to create, it was a perfect choice. I created a tiny Node Express app with a single POST “/notes” route, this was going to be the entry point from Hipchat. Once I got those two things hooked up and talking to each other I was sorted – because I’m a noob and couldn’t understand the Hipchat API Documentation fully, it took me bloody ages to figure out what was being sent from Hipchat and how I could grab that data to then start the next phase.

I found a video that really saved my skin and allowed me to move forward, the demo video used a tool called Requestbin that gives you a URL for Hipchat to POST the ongoing request. The Requestbin site then just displays every single piece of information in the request!! I was in business and I could navigate through the JSON, in the body of the request to get what I required.

Now that I had what I needed, I was able to use some Node modules to do most of the donkey work and get a working solution up and running quite quickly – At this point it was quite basic and very buggy, I think I broke it on the second request that I made to the application. I wasn’t actually thinking like a tester during the coding part, I was just trying to get something working. I actually don’t know why I didn’t plan and approach this like I would a normal project but you know, you live and learn.

 

mvp
My MVP

 

Bringing a tester onboard….

I needed to put my tester hat on and actually, think how I would use this application from a consumers point of view. I made a short bullet point list of possible ways that I can think that someone might use the tool or possible edge cases that I can mitigate against.

  • Entry point
    • /notes (I wanted this to be the only entry point)
    • /NOTES (I didn’t want this to return anything)
    • /NoTeS (I didn’t want this to return anything)
  • What could the user enter?
    • /notes valid number (/notes 12345)
    • /notes without a number
    • /notes number of digits (1, 2, many)
    • /notes and repeatedly entered 5 digit strings (/notes 12345 12345 12345)
    • /notes position in the sentence (“some words /notes 12345”)
  • Is the returned URL in the message a hyperlink? (Hipchat API auto picks up links, if in the correct message type “text/HTML”)
  • How do the other built-in slash commands behave (/code etc.)
  • What happens if the external GET resource is not available or returns the wrong data?

These are some of the questions that I was asking myself, I had many more but this is the general gist of the ideas in my head. The whole time that I was making changes to the code and then testing the outcomes I was thinking that I wanted to add something that would let me know what was going on behind the scenes of the application when the users were making requests. We recently had Richard Bradshaw come into our company to deliver his awesome Lego automation workshop, in this he mentioned “Giving your automation a voice” – This wasn’t test automation but I did want to know what was going on, it was only going to tell me if I actually told it too. I added log messages to the application at key points so that I could track what was actually happening rather than just seeing an output in a Hipchat room.

 

app_logging
Giving It A Voice

 

Another approach I was thinking about while testing was Ben Simo‘s FAILURE mnemonic if users were entering the wrong information, I wasn’t telling them (oops), It just didn’t return anything to the Hipchat room. That didn’t sit right with me and I used Ben’s heuristic to guide me, I wanted to give the users some feedback and let them know that “Something wasn’t quite right” with the data that they had inputted. As I could tell which Hipchat user sent the request, I grabbed that information from the JSON and made my returned messages (both good and bad) a bit more “friendly”. Manners don’t cost anything, as my Mum used to tell me.

 

release_notes_helper
Making It Friendly

 

error_handling
Appropriate Error Messages

 

Finding bugs in the wild…

I wanted to add an additional feature for users to specify a different environment when making the request (e.g. “/notes 12345 cloud4“), I initially defaulted the response to our main production node when users entered “/notes 12345“, as this matches the current behavior of the main internal release notes creation tool. I wanted to be a little bit more flexible, I had the opportunity to add extra functionality and let the users decide what the returned data was going to be. I added the code and rushed through the change, “It worked” so I left it and communicated out the new feature, giving an example of “exactly” how this worked. Great success! Maybe not…

 

cloud_request
Location Matching Feature

 

I’ve used Regular Expression in a number of places to capture certain things, I’m new to using Regex so I had a little help from the Regex101 site and also practised some of the Regex in the Chrome Dev Tools console (This was super useful!!). It turned out that my Regexfu is not so good and I omitted the “i” from the expression. meaning it was case sensitive so if someone entered “cloud4” it would return what they thought it would but “Cloud4” and “CLOUD4” wouldn’t return the same thing. From a users point of view, you would still get back a valid URL as it defaulted to a certain value if you didn’t enter exactly “cloud4”. I’ll get back to this issue.

Logging and Monitoring…

Using Heroku, I could access their built-in raw logs and could see what was happening while people started using the application. These were great and showed me what I wanted to see, the only trouble was, I had to go on to the Heroku site each time. At NVM we use Papertrail and I know that I could bolt on this log management tool to my Heroku app and feed the log messages into there. Great, I know this app and the search syntax so I was sorted….not quite, because we use in at NVM it annoyed me that I had to login and logout each time I want to see either the work logs or the ones that my app was producing.

Up stepped LogDNA – it was another option from the Heroku list, so I gave it a whirl. Luckily, the search syntax was very similar to Papertrail and it was extremely easy to get it set up in the way I wanted. It also had a feature where you can set up a notification on certain searches so that you get emailed when things start going wrong or just have something that lets you know who is using the app.

 

email_notification
Email Notification

 

Coming back to the Regex bug that I found earlier, I basically only saw this because of the logging that I had in place. I could see that a user had made a request, that should have failed (in my mind) but it returned the default value. I was able to spot the issue, fix it, test it and then deploy it again in super quick time without it impacting the users. Testability win!

Distribution…

I’ve now added this app to multiple Hipchat rooms and I’ve seen the usage steadily increase, which make it all worthwhile. I’ve also created an internal wiki page so that people can add it to any Hipchat room and it also explains some basic instructions with examples.

As well as doing this internally, I’ve knocked up a very simple bare-bones node app and placed this in a Github repository for anyone to use/rip apart etc.

You can grab the code here: https://github.com/DannyDainton/node-hipchatbot-basic

That was an epic read but hopefully, it makes sense and encourages other people to give things a go and try and improve a process that annoys you too!

Cheers!!

Creating Random Data on my Lunch Break…

A few days ago I read a great post by Alan Richardson called “How to write a simple random test data sentence generator” and i loved it, it’s exactly what I try and do all the time. In my current context, data is king, without data flowing through our service we effectively have no service. While testing stories, I create tools that do loads of little things but I have a selection of ones that create specific test data. Now, I’m not a developer, I don’t have any sort of development background but what I am, is an inquisitive learner.

I love writing code, not because I’ve been made too or I’ve been told too by my company so that I can automate all the things, I do it because it’s fun! I get a massive sense of achievement when I’ve had an idea for a tool, wrote some code and it’s worked!! It’s not the best looking and most efficient code but it’s my code and i’m proud of it!

So where was I, the random data generator – I read Alan’s post just before lunch and just before i tucked into my food i wrote a very basic app that spit’s out random data. All in all it took me less than 5 mins, it was done as a kind of exercise to prove that absolutely anyone can code something that creates test data. Don’t be afraid of just giving something a go, you might actually surprise yourself!

*This is wrote using Node.js but there’s probably tons of modules and libraries in your preferred language that will allow you to do the same thing*

We need to start somewhere…

In order to get your hands on this random data you’re going to have to download Node.js it won’t take very long and it’s so worth it, not only for this but for many other awesome things that you can create using Node.

Once you have it installed, type node -v into a command prompt or terminal. You should be presented with the current version that you have on your machine. If you’re not seeing the version, don’t panic, you might have to add the node directory to the path environment variable. You’ll find loads of step by step guides for this if you Google it.

node -v

node_version

Let’s do the code part!

So first things first, we need to create a new directory for your new and wonderful data generator to live. Open a command prompt or terminal (you may have one still open) and enter the following commands, as seen below. It’s entirely up to you what the root directory should be, after all, you’re doing this on your own machine.

mkdir random_data_generator
cd random_data_generator

*You don’t have to create this but I just like creating it so that i can give the app to other people*

Next, we need to create the package.json file this is the config file for your node app, it lists all the dependencies that you have in your project and also a bunch of other stuff you may need to add, as the app starts growing legs. You can read all about what this file is and what it does here but for now, we’ll crack on…

In the terminal enter the following and press Enter. Press enter through all the questions and type Y or Yes at the end.

npm init

Now we have that json file created we need to bring in the node module that’s going to do all the leg work for us and create the random data. Casual is a great helper module that rapidly speeds up the process of creating random data. There are a couple more great modules out there that do the same thing, like Chance.js and Faker.js but were going to use Casual this time because it’s the first one I thought of and I’m writing this post so that’s what you’ve got!

Install Casual by entering the following command into your terminal and pressing Enter. The –save in the command adds this module to the packages.json file, honestly, go and have a look if you don’t believe me.

npm install --save casual

When it has finished installing the module, leave the terminal open as you’re going to need it in a few mins.

Create a new file with your favorite text editor (Sublime Text, Notepad++ etc.) and save this in the new random_data_generator directory. For this example, I’ve named the file data.js but you can choose a name that’s more relevant to you.

In the file add the following code…

 // Dependencies
var casual = require('casual');

// Using the module to create random data
var number = casual.array_of_digits(n = 4);

var name = casual.first_name;

// Log the result
console.log(number);
console.log(name);

There are a couple of things going on here and I’ll explain (probably really poorly) what they are, so starting from the top, the first line after the comment brings in the Casual module so that we can use all of that goodness. Next, in the middle section, we’re using a couple of the in-built Casual functions that will create the random data and assigning these to the variables. Finally, we are just logging the output in the console. Very basic and extremely simple.

Using the terminal that you previously opened, type node data.js and press Enter to run the command. If everything has been setup correctly, you will be presented with some random data. It’s as easy as that.

data_output_02

 

If you create another file in the same directory and save this one as user.js I can show you a simple way of creating a set of basic user data. Add the following code below to the file and hit save again. To get an idea about what you can do with the Casual module, check out the documentation.

// Dependencies
var casual = require('casual');

// Custom Data Set

var password = '#A#B#C#D#E#F#';

casual.define('user', function() {
 return {
 Firstname: casual.first_name,
 Lastname: casual.last_name,
 Password: casual.numerify(password)
 };
});

// Write the result to the console

console.log(casual.user);

What this file is doing is creating a data set, defined and configured to your needs. Using the define function of Casual you can package this all together in a lovely little ball and do what ever you like with the output. On this occasion, I’m just logging it out to the console again. There are a few other things happening in this data, i’m using a couple of other inbuilt functions to create a different (very rubbish) password.

In the terminal type node user.js and press Enter to run the command. There you go…some more data for you!

data_output

Where do you go from here…

As you can tell this is very basic and the only thing it really does is print data out to the console – Not that helpful. This was only ever a 5 min task to quickly knock something together, we can expand this out to push the data to a certain file format that your system uses or feed the data into an already established test tool that you have been manually creating random data sets for, etc etc….I like the idea of creating Json files that can be added to apps like Bug Magnet so that you have the data there for you, while you’re doing an exploratory session.

We are only ever limited by our own imagination so it’s entirely up to you where you go from here but i’d love to hear about it so drop me a message!

*If you really can’t be bothered to create a tool yourself – Just Google “Random Data Generator” there’s bloody tons of them knocking about!!

Cheers for reading!

I now have a follow on post where you can put some of this beautiful data to good use…https://dannydainton.com/2016/09/28/everyone-loves-a-bit-of-bug-magnet/