nick evans – devblog

Updated 2008-12-10: added screenshot and video.

I’ve always been a little bit bemused by the default ruby test/unit and rspec output: a series of dots, one for each passing test (or example), and the dots become “F” characters in the event of a failure.  Most importantly: it’s simple, unambiguous about pass/fail, and easy to read at a glance, so no problem there.  You get a feel for the speed at which the tests are running, and you can visualize how many tests have run already.  And it’s easy enough to have them output in color (red or green), to make the errors visually pop out.  And it’s drop dead simple.  That’s good.

So what’s missing?

(for my tastes)

• Immediate Feedback: Yeah, the default output gives me an “F”, but what am I supposed to do with an “F”? I won’t know where and what that “F” was until the end of the entire test run, when the summary is dumped.  When running through a (functional) test suite that takes three minutes or even fifteen minutes, that just doesn’t work.
• Concise output: if everything is passing, I don’t need to see 4000+ dots displayed across my screen. It would be nice if the entire output would fit into a single line (when there are no failures).
• Percentage: A nice to have, not a need to have. But knowing how many tests have already run is generally not as interesting to me as how many tests are there total, and what percentage of them have run.
• ETA: This isn’t so important for the short test runs that I do over and over again during the BDD cycle… because I expect those tests to finish in under 10 seconds anyway.  But anything more than 10 seconds, and I get distracted. An ETA helps me limit my distraction.

So what do I do about it?

Basically, what I want is a progress bar and that the errors and warnings be displayed immediately. I also want warnings to be printed for slow specs. When using color, I want the entire progress bar printed in green if everything is good, yellow if there has been a warning, and red if there has been an error.

Fortunately, rspec makes it very simple to write a custom output formatter. So, a couple of weekends ago, I threw something together in about half an hour. I’ve used it ever since, and I’m pretty happy with it (I’ve tweaked it a little bit).

What does it look like?

A static image doesn’t really show off the progress bar very well, but here’s a screenshot (click thumbnail for full-size):

And here is a video, so you can see it in action (very low quality, sorry):

The code

http://ekenosen.net/bzr/rspec_support/compact_progress_bar_formatter.rb

It’s currently hosted in a bzr branch, but I’ll stick it up on github if anyone really cares about that.  Also, it depends upon the progressbar gem, so you’ll need to install that.

To use it

The location you download the script to is not important, but this is where I’m (currently) keeping it. You could also use bzr to pull down the entire branch.

$ gem install progressbar
 
$ mkdir -p ~/lib/rspec/
$ cd ~/lib/rspec
$ wget http://ekenosen.net/bzr/rspec_support/compact_progress_bar_formatter.rb
 
$ spec --require ~/lib/rspec/compact_progress_bar_formatter.rb -c -f Spec::Runner::Formatter::CompactProgressBarFormatter path/to/specs

I’m currently using some bash aliases to keep the spec command line nice and short. Perhaps there’s a better way?  In ~/.bash_aliases:

alias  spb='spec        --require ~/lib/rspec/compact_progress_bar_formatter.rb --format Spec::Runner::Formatter::CompactProgressBarFormatter --color'
alias sspb='script/spec --require ~/lib/rspec/compact_progress_bar_formatter.rb --format Spec::Runner::Formatter::CompactProgressBarFormatter --color'

For rake tasks, I use a spec/local_spec.opts (spec/spec.opts is checked into version control as a default).

Nitpicking

I cheated: I didn’t develop it BDD-style.  I intend to fix that soon by rewriting it spec-first; certainly before adding any more functionality. TATFT indeed.

I’d really like to be able to send the threshold (for reporting of slow specs) in on the command line.  It might be easy to do, I haven’t looked into it… I’m currently just editing the file every time I want to change the threshold (e.g. normal specs run with a threshold of 0.1 sec, functional specs run with a threshold of 5 or 10 seconds).  Messy.

Also, it would be nice if the command line didn’t need to be so long… perhaps a supported gem/plugin system for rspec?  If that were easily doable, then I would probably package this up into a gem.

I had to break into the progressbar’s internals a little bit… it didn’t publically expose all of the functionality I wanted.  Still, that’s better than creating my own progressbar library.