Thursday, October 23, 2008

BDD-style feature tests using IronRuby and RSpec/Cucumber

Introduction

BDD (Behavior Driven Development) is a specialization of TDD (Test Driven Development) that encapsulates a set of best practices espoused by the most successful TDD gurus.  The main theme is focusing on behavior rather than implementation in your tests.  A great tool for verifying behavior (writing BDD-style tests) is RSpec, and now it can be used to test .NET code.

To get more information on BDD, read Dan North's Introduction to BDD.  To get some background information on TDD, read Scott Bellware's classic Red-Green-Refactor post or read Kent Beck's TDD By Example (this one's on my to-do list, but I've heard great things about it).

RSpec is a unit testing tool for Ruby that provides an internal DSL which supports nice BDD-style specifications.  An example of that syntax is below.

 

# bowling_spec.rb
require 'bowling'  

describe Bowling do
  before(:each) do
    @bowling = Bowling.new
  end

  it "should score 0 for gutter game" do
    20.times { @bowling.hit(0) }
    @bowling.score.should == 0
  end
end

RSpec also provides an external DSL (the RSpec Story Runner) that allows you to create executable plain-English feature documentation in the Given/When/Then format typical of BDD tests.  This tool has been broken out into its own project called Cucumber, and an example of the syntax is below (from the RSpec website).

 

Feature: transfer from savings to checking account
  As a savings account holder
  I want to transfer money from my savings account to my checking account
  So that I can get cash easily from an ATM 



  Scenario: savings account has sufficient funds
    Given my savings account balance is $100
    And my checking account balance is $10
    When I transfer $20 from savings to checking
    Then my savings account balance should be $80
    And my checking account balance should be $30
 
  Scenario: savings account has insufficient funds
    Given my savings account balance is $50
    And my checking account balance is $10
    When I transfer $60 from savings to checking
    Then my savings account balance should be $50
    And my checking account balance should be $10

 

As the RSpec website says "Each Given, When and Then is a Step. The Ands are each the same kind as the previous Step. Steps get defined in Ruby like this (detail left out for brevity) in steps.rb (in the same directory in this example):"

 

Given /^my (.*) account balance is \$(\d+)$/ do |account_type, amount|
  create_account(account_type, amount)
end

When /^I transfer \$(\d+) from (.*) to (.*)$/ do |amount, source_account,
target_account|
  get_account(source_account).transfer(amount).to(get_account(target_account))
end

 

Then /^my (.*) account balance should be \$(\d+)$/ do |account_type, amount|
  get_account(account_type).should have_a_balance_of(amount)
end

 

Getting RSpec and Cucumber set up to work with .NET

The first thing you need to do to start verifying the behavior of .NET code using RSpec is to install IronRuby.  In theory, it's also possible to use RSpec to test .NET code by using jRuby and Java's interoperability with .NET, and in fact there's an example of that approach that comes bundled with Cucumber, but I thought this was one too many layers of interoperability (jRuby -> Java -> .NET) for an approach I am going to attempt to use to verify behavior in the majority of code I write.  Also, Cucumber's web site says "When IronRuby matures it can be used to 'test' .NET code too", so I took that as a hint that the jRuby way might be problematic in the long run.  Finally, I wanted an excuse to play with IronRuby :-)  To get the latest version of IronRuby, you must install TortoiseSVN and then do an SVN Checkout from http://ironruby.rubyforge.org/svn/trunk.  Open IronRuby.sln in Visual Studio (telling it to "Load Projects Normally" if prompted) and Build Solution.  There is a ZIP of IronRuby you can download from rubyforge also, but that didn't work too well for me, so I wouldn't recommend it.  (I suspect it's a significantly outdated release.)

In order to obtain Cucumber and all its dependencies, download and install the latest version of the Ruby One-Click Installer (henceforth referred to as regular Ruby).  At the command line (from any folder), type "gem Cucumber" and answer "Y" when it asks you to install each of the dependencies.  The reason that regular Ruby (rather than IronRuby) is used for this step is because I was not able to get RubyGems to work on IronRuby.

Next, copy the contents of all the gems you just downloaded from C:\ruby\lib\ruby\gems\1.8\gems (assuming you installed regular Ruby to C:\ruby) to C:\Projects\IronRuby\trunk\lib (assuming you checked out the IronRuby trunk to C:\Projects\IronRuby\trunk).  A list of the gems you will need to copy is as follows:

cucumber
hoe
polyglot
rake
rubyforge
rspec
term-ansicolor
treetop

Note that you will specifically want to copy only the contents of the "lib" directory of each of these gems.  (For example, C:\ruby\lib\ruby\gems\1.8\gems\treetop-1.2.4\lib\treetop.rb will be copied to C:\Projects\IronRuby\trunk\lib\treetop.rb and similarly the C:\ruby\lib\ruby\gems\1.8\gems\treetop-1.2.4\lib\treetop folder will be copied to C:\Projects\IronRuby\trunk\lib\treetop.)  Copying the contents of the "lib" folder of each of these gems was the only way I could manage to get IronRuby to recognize all of them at the same time.  I have heard that setting the GEM_PATH environment variable to the location where you have put your gems will enable IronRuby to recognize them, but that didn't work for me, which necessitated the kludgy step I just described.

Next, you will need to modify a few files within RSpec and Cucumber to get them to work with IronRuby.  the modifications are as follows (paths are relative to C:\Projects\IronRuby\trunk):

  • In lib\cucumber\formatters\pretty_formatter.rb, find the "source_comment" method, comment out the body of it, and add simply two double-quotes (an empty string) as the new body of the method.  This is because the executable file location doesn't seem to be available in IronRuby.  (The commented out code is what would normally print the name of the file and the line number of each code definition referred to by the specification.)
  • In lib\cucumber\formatters\ansicolor.rb, comment out the first line that says "gem 'term-ansicolor'".  (I wasn't able to get ANSI color to work for the output from Cucumber.)
  • In lib\cucumber\tree\step.rb, find the "execute_in" method and then find the following code:

method_line_pos = e.backtrace.index(method_line)
if method_line_pos
  strip_pos = method_line_pos - (Pending === e ? PENDING_ADJUSTMENT : REGULAR_ADJUSTMENT)
else
  # This happens with rails, because they screw up the backtrace
  # before we get here (injecting erb stactrace and such)
end
format_error(strip_pos, proc, e)

  • and change it to:

if e.backtrace
  method_line_pos = e.backtrace.index(method_line)
  if method_line_pos
    strip_pos = method_line_pos - (Pending === e ? PENDING_ADJUSTMENT : REGULAR_ADJUSTMENT)
  else
    # This happens with rails, because they screw up the backtrace
    # before we get here (injecting erb stactrace and such)
  end
  format_error(strip_pos, proc, e)
else
  e.extra_data = format_error2(proc, e)
  raise e
end         

  • In lib\spec\expectations\errors.rb, add two properties to the "ExpectationNotMetError" class as follows (this step and the last step are necessary because the file name and line are not included with the Exception message in IronRuby as they are in regular Ruby):

def extra_data=(value)
  @extra_data = value
end

def message
  to_s + "\n" + @extra_data
end

 

Testing .NET code

To complete the test, I created a simple C# source file to test called "Accent.cs" as follows:

namespace TestLibrary
{
    public class Accent
    {
        private readonly string _stateAbbreviation;

        public Accent(string stateAbbreviation)
        {
            _stateAbbreviation = stateAbbreviation;
        }

        public string PronounceWord(string word)
        {
            if (_stateAbbreviation == "MA")
            {
                switch (word)
                {
                    case "bar":
                        return "bah";
                    case "dollar":
                        return "dolla";
                }
            }

            return word;
        }
    }
}

I compiled that source file into an assembly called "TestLibrary.dll" and copied it (and TestLibrary.pdb) to a new folder: C:\Projects\IronRuby\trunk\lib\lib.

Next, I created a file called cucumber.yml in C:\Projects\IronRuby\trunk\lib with the following contents (copied from the "calculator" example provided in Cucumber):

default: --format pretty features

I also created another file (also copied from the "calculator" example) called Rakefile with the following contents:

$:.unshift(File.dirname(__FILE__) + '/../../lib')
require 'cucumber/rake/task'

Cucumber::Rake::Task.new do |t|
  t.cucumber_opts = "--profile default"
end

Next, I created a file called pronunciation.feature (a specification) in a new folder: C:\Projects\IronRuby\trunk\lib\features with the following contents:

Feature: Pronunciation
  In order to gain the trust of a customer
  As a sales representative
  I want to pronounce words in the dialect of the customer

  Scenario: Pronounce a word
    Given My client lives in MA
    When I pronounce bar
    Then the word should be pronounced bah

  Scenario: Pronounce a word
    Given My client lives in CA
    When I pronounce bar
    Then the word should be pronounced bar

  Scenario: Pronounce a word
    Given My client lives in MA
    When I pronounce dollar
    Then the word should be pronounced dolla

  Scenario: Pronounce a word
    Given My client lives in CA
    When I pronounce dollar
    Then the word should be pronounced dollar

Next, I created a file called proncuation_steps.rb (defining the steps in the specification above) in a new folder: C:\Projects\IronRuby\trunk\lib\features\steps with the following contents:

require 'spec'
$:.unshift(File.dirname(__FILE__) + '/../../lib')
require 'mscorlib'
require 'TestLibrary'

Before do
end

After do
end

Given "My client lives in $state" do |state|
    @accent = TestLibrary::Accent.new state
end

When /I pronounce (\w+)/ do |word|
  @result = @accent.PronounceWord word
end

Then /the word should be pronounced (.*)/ do |result|
  @result.to_s.should == result.to_s
end

Finally, I created two files in C:\Projects\IronRuby\trunk\build\debug. The first of them is icuc.rb, which contains the following:

require 'cucumber'
require 'cucumber/cli'

Cucumber::CLI.execute

The second file is icuc.bat, which contains the following:

@echo off
set IRONRUBY=c:\Projects\IronRuby\trunk
pushd %IRONRUBY%\lib
%IRONRUBY%\build\Debug\ir %IRONRUBY%\build\Debug\icuc.rb
popd
set IRONRUBY=

These two files are the equivalent of the "cucumber" and "cucumber.cmd" files in C:\ruby\bin.  You can now type "icuc" at a command prompt and it will run the Cucumber test you just created, which should pass!  I chose the name "icuc" so that it wouldn't conflict with the "cucumber" command in regular Ruby.  Happy testing, er... I mean verifying! ;-)

If you have any trouble getting this to work for you, or if you know of a better way to do it, please leave a comment!  I know this method is less than ideal, so I'm hoping one of you can help me improve it. :-)

posted @ Thursday, October 23, 2008 12:54 AM | Feedback (7)