Cucumber 101

My primary reason for using Cucumber in a recent automation initiative was to try and bridge a technology-business divide that existed in my workplace by giving the non-technical stakeholders a larger sense of involvement in testing with a (longer term) view to them actively contributing to future test specs by using the plain English approach that Cucumber does so well.

However, when I first started looking into Cucumber, there were a multitude of references out there but as I was also learning about Ruby, Watir and RSpec at the same time it became a little difficult to see the wood for the trees because they tended to talk about all of these things without being clear about where one thing stopped and another began.

What I’d like to do with this post is present a simple Cucumber 101 tutorial, letting cucumber guide us through the creation of a set of ‘scenarios’  (this is what Cucumber calls ‘tests’ or to be more precise, ‘checks’ – this is automated after all).

1. Cucumber Basics

Cucumber is a command line tool and when you run it, it looks for plain-language text files which typically contain one or more features, and within each feature, one or more scenarios to test. Within each scenario you have a number of steps that Cucumber works through. There is some syntax involved so that cucumber can understand what you’ve written and this is known as Gherkin. I’ll highlight this in the examples below.

Cucumber Schematic

So if you were to see the directory structure required for a simple cucumber project, it would look something like this:

Cuke001

2. Installation Prerequisites

This tutorial assumes you have a copy of ruby, rubygems (or equivalent), watir-webdriver, rspec and of course cucumber installed on your target system.

3. Create a Scenario

Let’s create a new Cucumber project (a directory, called ‘Cuke’) and try running cucumber in this empty directory.

Wait, how do we run cucumber? Simple, type ‘cucumber’ (Note: We can specify arguments but in the absence of any cucumber assumes you want to run everything – more on that later).

Cuke002

So we have no features or scenarios. Cucumber is looking for a features directory. Let’s create one as it suggests. Once we’ve done that, let’s try running cucumber again.

Cuke003

So now we see that although cucumber found a features directory, there were no feature files and consequently no scenarios to run, hence the “0 scenarios” and “0 steps” references.

Let’s create a scenario in a feature file under the features directory, and let’s call the feature file karate.feature. Here’s the content:

Cuke004

For those of you unfamiliar with karate, Hironori Ohtsuka was the founder of the Wado-Ryu style of karate, and our scenario is checking for the presence of the founder’s surname in the Google results on a search for ‘Wado’.

You can see the Gherkin grammar highlighted in blue above – these must be used when writing feature files – the key Gherkin words include – Feature, Background, Scenario, Scenario Outline, Scenarios (or Examples), Given, When, Then, And (or But) and a couple of other symbols.

Now let’s run cucumber again with the new feature file in place:

Cuke005

So, cucumber is now picking up our new feature file. Great!

However, all of our scenario steps are as yet undefined.

4. Add Step Definitions

So, two things about the above output. Cucumber is telling us that our steps and scenario are as yet undefined (the yellow text) and that as yet, it doesn’t know which programming language we intend to use to implement our scenarios (the red text). Let’s take our lead from Cucumber and copy the suggested steps into a new step definition file, say step_karate.rb (The .rb suffix indicates a Ruby file). Like so…

Cuke006

and run cucumber again

Cuke007

You may notice that the red text we saw in the ‘undefined’ error above has now gone. Why? We’ve put a Ruby (.rb) step definition file in place so Cucumber now knows we’re intending to use ruby to implement the scenario. So now we can map all the steps in our scenario to a step definition file, and cucumber helpfully provides the file and line number of each step it finds (this comes in useful later when you have large suites of scenarios) You’ll also notice that Cucumber has deemed the first step as ‘pending‘ (I.e. to be implemented) – this is due to the pending statement in the step definition. This acts as a placeholder so you can write some proper code for each step when you are ready. Cucumber takes a sequential approach to scenario steps and the subsequent steps are deemed ‘skipped

5. Implement Step Definitions

Let’s implement our first step definition now. We’ll take out the pending statement and put in some Ruby code that will start a web-browser.

Cuke009

So for our first step “I navigate to Google” we’ve removed the pending statement and inserted two lines of Ruby/Watir-Webdriver, to start a browser (Firefox) and to navigate to Google.

Note: I’ve included a Ruby ‘require’ statement in the step definition file. There are tidier ways of doing this since the key to maintainability of any cucumber suite lies with the step definition files. We’ll tidy this up a bit later

Let’s run cucumber again

Cuke010

Now we see the first step passes (green), the second step is pending (yellow), and the third step is skipped (blue). Progress! You’ll also notice a browser opened in front of you (at Google) that hasn’t been closed – that’s because we didn’t close it in our code (yet). For now, kill the browser – we’ll tidy that up later. Let’s implement the remaining steps as shown below:

Cuke011

In the second step, we create variables for the Google search field and button, then enter the value ‘Wado’ into the search field and click on the button. In the third step we check that the results page has loaded and then perform another check to see if the text  ‘Ohtsuka’ is present in the results page. If it’s there, the step will pass, and if it’s not the step will fail. Lastly, we close the browser instance.

6. Passed & Failed Scenarios

So, if we run cucumber once again with all our implemented steps:
Cuke012
Whoa! 3 passed steps and 1 passed  scenario. Nice!
However, in order to see what a failed scenario looks like, let’s deliberately alter the string literal in the last step  ‘Ohtsuka’ to ‘Octavias’ as we don’t expect that to appear in the page, and run cucumber again:
Cuke013
So you can see from the above the scenario has now failed after being unable to locate ‘Octavias’ in the results page. If you run this you’ll notice that the browser stayed open which means the @browser.close statement was never executed. Hmm, messy. Let’s tidy this up a bit…

7. Tidy Up – Environment, Hooks, etc

First of all, let’s get rid of that ‘require’ statement in the step definition file and put it somewhere more appropriate. We’ll create a new directory under features, called support, and under there we’ll create an environment file, env.rb and add two lines

Cuke014
The first line extends the path that Cucumber picks up to include a Cuke/lib (we haven’t created that yet)
The second line includes a Ruby file called myClasses.rb (we haven’t created that either)
Let’s create those too.
Cuke015
So our require watir-webdriver statement lives here, we can remove it from step_karate.rb
Next, let’s add another file to the support subdirectory, called hooks.rb and use the Before and After methods to define actions that should be taken before and after each and every scenario. Like, starting up and closing down a browser.
Cuke016
Now we can remove those statements from step_karate.rb as well.
If we re-run the scenario this time, it will still fail but the browser will be closed down regardless of the outcome.

8. HTML Formatters

You can also direct cucumber output to  an HTML file using the default HTML formatters. by running the command:

cucumber -f html >output.html

The contents of output.html then look like this:
Cuke018

So as you can see the failed step is highlighted in red, as is the scenario, as is the cucumber header (if one scenario out of many fail then the header is coloured red accordingly).

Now let’s change the string ‘Octavias’ back to ‘Ohtsuka’ and re-run

Cuke017

The Scenario headers in these HTML files can be collapsed or expanded individually (left click on it) or along with everything else in the page (use ‘Collapse All’/’Expand All’ in the header)

9. Multiple (Similar) Scenarios – Placeholders

So our scenario checks the presence of the karate founder’s surname in the results when we search for the style name. What if we wanted to run a similar scenario for other styles of karate, say:
  • Wado-Ryu (Hironori Ohtsuka)
  • Shotokan (Gichin Funakoshi)
  • Goju-Ryu (Chojun Miyagi)
  • Kyokushinkai (Masutatsu Oyama)
We could of course write separate scenarios and create new step definitions (primarily for steps 2  and 3, step 1 could be reused). The trouble there is that when you have a lot of scenarios which do the same thing with different arguments, it becomes laborious to wade through the results and difficult to see the uniqueness of each scenario.
So let’s consider refactoring our scenario to become a scenario outline – that way we only have to specify our steps once, and use placeholders to identify the unique values we wish to use within each iteration. Here’s what the refactored feature file looks like:
Cuke019
So the placeholders are the elements in the angled brackets in the steps, <karateStyle> and <masterSurname>. The Examples section (Note, you could also use the text ‘Scenarios:’ here instead of ‘Examples:’) is simply a table with the placeholder names (minus the angled brackets) as headers and the values you wish to use below them.
The scenario outline is useless without the examples and the four examples now constitute a single scenario.
Let’s take a look at the refactored step definition file:
Cuke020
The (\w+) elements are shorthand character classes and stand for a word character, specifically [A-Za-z0-9]. The value is read into the variable (karate or surname) which we can then use in place of the string literals (‘Wado’ and ‘Ohtsuka’) we had previously.

10. Tagging

Tagging is easily one of the most powerful aspects of cucumber and for me, the thing that made it so versatile when dealing with a large number of scenarios. A tag in cucumber is simply a text string preceded by the ampersand character, and we generally use them in the feature file by placing them before Gherkin keywords such as Scenario Outline, Scenarios, Examples, etc. You can have more than one tag for the same scenario, just separate them with spaces.
Tags can represent anything that’s meaningful to you – the name of the scenarios (e.g. @scenario1, @scenario2, etc.) , the performance of the scenarios (e.g. @quick, @slow, @glacial), the frequency of the scenarios (e.g. @hourly, @daily, @weekly, etc.) – anything that makes it easier to slice and dice your scenarios into meaningful cross-sections that make execution a breeze.
Let’s refactor our feature file with some tags:
Cuke021
We can run cucumber and pass in some arguments to tell it to run everything associated with a particular tag by using a command like

cucumber -t <tag>

where <tag> is the tag name. So in the case above if we ran:
  • cucumber -t @all  (This would run everything – all four scenarios)
  • cucumber -t @daily (This would run the top two scenarios for Wado and Shotokan)
  • cucumber -t @monthly (This would run the bottom two scenarios for Goju and Kyokushinkai)
  • cucumber -t @wado (This would only run the top scenarios for Wado)
You get the idea, right?
So there you go, 10 steps to cukey heaven. Enjoy…

Bridging the Gap – Automation

A few months ago I was mulling over the best way to approach a proposed automation initiative I had proposed when I came across a well written and highly informative blog by Elisabeth Hendrickson that helped me focus on my objectives and narrow my search criteria. Our situation is largely similar to a lot of other organisations in that we have a web-based product with a plethora of checks that are ripe for automation. We’re understaffed and overworked (aren’t we all?), so my primary aims included:

  • A way of executing automated functional checks on existing features quickly for each candidate build
  • Transparency across platforms and browsers (Windows, MacOS, Firefox, Chrome, IE, etc.)
  • Allow the sapient testers to focus on testing and discovering new information about the product
  • An efficient way of communicating automation results to all stakeholders without overwhelming them with data
  • A way of involving the non-technical stakeholders so they feel part of the technical side of things (without having to be technical)
  • Providing a way of capturing screenshots that allow manual inspection of aesthetics (also useful for demos, presentations, bug reports, etc)
  • Build some bridges between technology-business, hopefully stimulating further collaboration through visibility

Implementation

I’m a big fan of scripted languages and tend to go for Perl more often than not, but Ruby was catching my eye for this particular adventure. I love the ability to simply  “write’n’run” code without having to deal with compilation, linking, etc or having to invest in expensive IDEs to bring projects to life. Ruby has elegant syntax and a wealth of support in the online community, plus the inclusion of IRB (Interactive Ruby Shell) which was marvellously easy to use when experimenting with new code.

I just wanted an intelligent editor and a command line. As far as editors go, vim  was always an option but as I was going to implement this on a Mac, I plumped for BBEdit which, as Bare Bones Software suggests, “it doesn’t suck”. This afforded me the ability to switch between multiple files very quickly (you can do this with vim too, but I liked the presentation and the reviews were favourable).

Drivers

While the implementation was a no-brainer, the choice of driver – i.e. the thing that would manipulate my web interface was swayed somewhat by my previous experiences with Selenium. Like most people, I had used the Firefox plugin and played with the IDE record/playback functionality and found that the export utility to Ruby was less than perfect. Selenium RC (as it was called at the time) wasn’t much better (for me at least) and I encountered some real problems when trying to interact with javascript elements. It’s much improved now by all accounts but I was already searching for alternatives (a bias on my behalf, I guess).

The other obvious choice was Watir which seemed like a natural fit to me, but after reading a series of posts from Alister Scott, I chose to go with Watir-Webdriver instead. I particularly enjoyed his ‘meaty example‘ post, comparing Ruby automated APIs. The syntax looked elegant as well as affording me the chance to make the code maintainable (as well as readable). In principle, this choice would support IE, Firefox and Chrome across Windows and MacOS (with the possibility of Opera too). It also allowed me to capture ‘full screen’ screenshots (as opposed to ‘visible area’ screenshots) – something which my previous endeavours with Selenium caused more pain by not working as I’d hoped.

Frameworks

There are a number of frameworks to choose from but the first couple I looked at were a bit too ‘codey’ for my non-technical stakeholders, namely Test::Unit and RSpec. I even bought a copy of The Rspec Book but in doing so ended up reading and digesting the chapters within that talked about Cucumber. A soon as I saw this I knew it would give me the best chance of drawing in the non-technical stakeholders. I ended up buying The Cucumber Book as well, but not until I was most of the way through implementation – thankfully there are lots of great online references, tutorials, and examples to learn from on the web. The key things about Cucumber that won me over were:

  • Plain text descriptions
  • Nifty results presentation with the default Cucumber HTML formatter
  • Even niftier results presentation with a customised Cucumber HTML formatter
  • Ability slice and dice the automation suite using Cucumber tags – a very powerful feature

Now, granted we don’t do any sort of BDD at our place, but that didn’t stop me from choosing Cucumber (nor should it, IMHO)

Move to the current day, the initiative is complete and has been well received so far.

I’ll elaborate more on the details in later blog posts.

Follow

Get every new post delivered to your Inbox.