Wednesday, December 4, 2013

The Virtual Clock Test Pattern

You can have a hard time unit-testing code that depends on the system clock. This article describes both the problem and a common, repeatable solution.

[Note: This is an article I wrote in 2006. I'll publish it here because people sometimes still quote it, and the original version fell off the Internet. (But the older version, with examples in Java instead of Ruby, can still be found on the Wayback Machine).]

 

The problem


To see what the Virtual Clock is about, we can dive into a simple example. We'll code this in Ruby, but you should be able to understand it even if you're not a rubyist – and you'll learn some Ruby along the way.

We're building a minimal scheduling system where you can create tasks and execute them later. Each task has a maximum age, after which it expires and cannot be executed any more. Here's the Task class:

class Task
  def initialize(max_age, &action)
    @max_age = max_age
    @action = action
    @time_of_birth = Time.now.to_i
  end

  def age
    Time.now.to_i - @time_of_birth
  end

  def execute
    @action.call if age <= @max_age
  end
end

The method initialize is Ruby's equivalent of a constructor – it gets called when you write Task.new to create a task. The first argument of initialize is the maximum age of the task, in seconds. The argument is stored into a variable named @max_age. The @ prefix means this is an instance variable (you might be used to call these “object fields”). Ruby being a dynamic language, you don't have to declare the variable anywhere else: you just assign something to it, and it springs into existence.

The second argument of initialize is not really an argument – it's a block. Ruby blocks might look foreign to you. They allow you to write things like:

t = Task.new(10) { print “Executing, sir!” }

The curly braced thing is the block. It contains the action that we want the task to execute. From inside the method, it looks like an argument prefixed by an ampersand. From outside, it's more like a snippet of code that's attached to the method call. It's not executed immediately. Instead, Ruby internally converts it to an object, and we store it into the @action instance variable to execute it later.

The last instance variable, @time_of_birth, is initialized with the current time. Time.now gets the time from the system clock, and the method to_i converts it to an offset in seconds from a conventional date. The method age uses the same instruction to find out how old the task is.

Finally, the method execute does the real work: it executes the stored block by invoking its call method. This is only done for tasks whose age is less than @max_age – cranky old tasks just ignore your attempts to execute them. It's idiomatic Ruby to put the if condition at the end when the body is a single statement.

This is a smart little class, but we should have written a test for it. We haven't been good Test-First coders, have we? Better late than never, so let's focus on that test. But now we have a problem: to test this code, we need to be sure that some tasks expire before we execute them, and some others don't. That's not easy. We could insert pauses into our test by calling Kernel.sleep, but that would slow down the tests. It would also make it very difficult to test edge conditions. Worse still, as the code gets more complicated, our tests might become non-deterministic. You probably know the problem: sometimes the test code runs slightly slower for any reason, and the tests fail randomly. There is nothing worse than a test that fails 10% of the times.*

Unfortunately, we cannot control system time. Or, can we?

 

A solution


We need to control time itself. We can do this by defining two separate clocks:

class RealTimeClock
  def time
    Time.now.to_i
  end
end

class VirtualClock
  attr_accessor :time

  def initialize
    @time = 0
  end
end

The RealTimeClock returns the current system time when you call its time method. The VirtualClock doesn't care about the system clock at all. Instead, it returns the value of its time attribute, which is initialized to zero and can be modified by the clock's clients (attr_accessor just tells Ruby that we want an object attribute named time, and its value is stored into an instance variable named @time).

Now we can modify the Task class to use one of our brand new clocks:

class Task
  def initialize(max_age, clock = RealTimeClock.new, &action)
    @max_age = max_age
    @clock = clock
    @action = action
    @time_of_birth = clock.time
   end

  def age
     @clock.time - @time_of_birth
  end

  def execute
    @action.call if age <= @max_age
  end
end

You can pass the clock in when you create the Task. If you ignore the argument, it will be assigned a RealTimeClock by default. Now it's easy to write a solid test:

require 'test/unit'

class SchedulerTest < Test::Unit::TestCase

  def test_active_tasks_do_something
    clock = VirtualClock.new
    executed = false
    task = Task.new(0.2, clock) { executed = true }
    clock.time += 0.2
    task.execute
    assert executed
  end
end

We created a task that uses the Virtual Clock, and instructed it to change the value of the executed flag. Then we used our own little time machine to set the current time and check that the action associated with the task is actually executed. The test for expired tasks is even simpler:

def test_expired_tasks_do_nothing
  clock = VirtualClock.new
  task = Task.new(0.2, clock) { flunk }
  clock.time += 0.21
  task.execute
end

The block associated with this task is a call to flunk, a test assertions which always fails. We're simply testing that this action is never called, and flunk is never executed. Behold the Green Bar!

 

It all boils down to...


To write good tests, we need lots of control over our test environment. We must be able to set it up exactly as we like. If a piece of code relies on non-deterministic behaviour, then we are in trouble.

The system clock is non-deterministic by nature. It's an important system property, but we cannot control it. The Virtual Clock pattern gets around this by replacing the system clock with something that we can actually control.

Therefore:

Don't access the system clock directly. Instead, wrap calls to the system clock into a Clock object, and replace it with a Virtual Clock for testing.

 

More ideas


Global clocks – If you use this pattern, you can end up passing clocks all around the place. Some people dislike this, and consider it a case of tests polluting production code. An alternate solution is a singleton clock with global access. You can make it a Real Time Clock by default, and switch to a Virtual Clock for testing. But be careful: it's safe to have a global Real Time Clock, since this is a read-only object – but the Virtual Clock isn't. I was burned by this approach when I forgot to reset the global Virtual Clock after a test, and was punished by a mysterious failure in the following test.

Not only for testing – The Virtual Clock decouples the concept of “time as an input” from “real time”. Time becomes a variable like any other. This can be useful for things other than testing. For example, you might want to simulate a process over a long time span. Or maybe you have a piece of code that processes historical data, and you want to trick it into working at a different time than “right now”.

Clock supertype - The VirtualClock and the RealTimeClock of this article don't need to share any special relationship. In dynamic languages such as Ruby and Python, it's enough that both classes implement a time method. Any piece of code that relies on time alone will gladly accept any object that implements this method (this is known as duck typing). In Java or C#, the clocks need to share the same explicit type to get this kind of polymorphic behaviour. You'd probably do this by defining a common Clock interface.

A virtual family – Instead of a Virtual Clock that just counts seconds, you might want to define a Virtual Date to abstract calendar dates. You can also adapt this pattern to deal with any non- deterministic entity, such as random number generators or external device drivers.

Related patterns – In pattern-speak, a Virtual Clock is an example of a Test Double - more specifically a Fake Object. To make objects aware of the Virtual Clock, you can use any kind of Dependency Injection. In this article, we used Constructor Injection to pass the Clock around.

 

Known uses


Martin Fowler mentioned that he always uses indirection on the system clock. He touches on the subject when he describes the Time Point pattern.

Prevalence systems such as Prevayler use a Virtual Clock to guarantee deterministic behaviour.

Real-time coders routinely simulate time. John Carmack used this technique to test its Quake 3 game engine.

There are many more examples of Virtual Clocks around. This is a common pattern. [2013 update: There are many more examples available today. The Timecop gem is one of the current popular implementations of this pattern in Ruby.]

 

Thanks to...


The following people helped me review this article, gave me comments and suggestions, or just pointed me to useful material: Kent Beck, Emmanuel Bernard, Roberto Bettazzoni, David Corbin, Chad Fowler, Martin Fowler, Patrick D. Logan, Dan Palanza, J. B. Rainsberger, Andrea Tomasini, Marco Trincardi, Andrew Wall.

* On second thought, a test that fails 5% of the times is probably worse than that.

Sunday, December 16, 2012

Sssmoke - When even Sinatra is too much

Sometimes you don't need a web framework, no matter how light - you just want to slap a Ruby script or two on a web server. Enter Sssmoke:
gem install sssmoke

Put your erb templates in a directory, then run them in a web server by typing:
sssmoke

A template named foo.erb will get the URL http://localhost:8888/foo.

Advanced options for powah usahs:
sssmoke directory_name      # sssmoke templates from another directory
sssmoke template_name.erb   # sssmoke a single template at http://localhost:8888/

That's all. (As to the reason why it's called "Sssmoke"- that story would be better told in person.)

Saturday, August 11, 2012

What Good Error Messages Look Like

Good error messages do three things:
  1. They tell me what's wrong and how to fix it.
  2. They stand out.
  3. They soothe my soul.
A teammate just sent me some output from Maven, a popular package manager/project manager/kitchen sink for Java. She spotted a warning, well-concealed near the top of hundreds of lines of infodump. I had to format it to make it readable.

[WARNING] Some problems were encountered while building the effective
          model for [our_project]
[WARNING] 'dependencyManagement.dependencies.dependency.(groupId:
          artifactId:type:classifier)' must be unique: org.jboss.resteasy
          :resteasy-jackson-provider:jar -> version 2.3.2.Final vs 2.3
          .0.GA @ com.[my_customer].[our_project]:[our_project]
          :0.17.Beta-SNAPSHOT, /Volumes/Workarea/trunk/pom.xml, line 272,
          column 16
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they
          threaten the stability of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support
          building such malformed projects.

Gotta love the "please follow us, citizen" tone of those last few lines. I'll tell you what, Maven: either the stability of my build is really being threatened, and then you shouldn't hide this information so carefully - or it's not, and then you shouldn't be such a picky jerk. And by the way, your mother is way more malformed than my project. Sheesh.

Compare that to Homebrew, a package manager for OS X. After an operating system upgrade, I ask it whether there are any problems with my setup:

~$ brew doctor

Instead of dumping the Library of Congress to my terminal as Maven would, Homebrew gives me a clear screen with three well-formatted warnings. First one:

Warning: Some keg-only formula are linked into the Cellar.
Linking a keg-only formula, such as gettext, into the cellar with
`brew link f` will cause other formulae to detect them during the
`./configure` step. This may cause problems when compiling those
other formulae.

Binaries provided by keg-only formulae may override system binaries
with other strange results.

You may wish to `brew unlink` these brews:

    libxml2
    libxslt

I brew unlink the two libraries as instructed. Fixed. Second warning:

Warning: Some installed formula are missing dependencies.
You should `brew install` the missing dependencies:

    brew install autoconf automake libtool

Run `brew missing` for more details.

Command copy-pasted, libraries installed, problem fixed. Third warning:

Warning: /usr/bin occurs before /usr/local/bin
This means that system-provided programs will be used instead of those
provided by Homebrew. The following tools exist at both paths:

    bashbug
    clusterdb
    [more stuff]
    
Consider amending your PATH so that /usr/local/bin
occurs before /usr/bin in your PATH.

I spend a few minutes fixing the PATH. Let's try again:

~$ brew doctor
Your system is raring to brew.

That's why Maven fills my soul with darkness and desperation, while Homebrew overflows my heart with love and unicorns.

Tuesday, October 11, 2011

Windows 8: solving the wrong problem

In the 80s and the 90s, we had a Holy Grail: cross-platform compatibility. We wanted to write our software once, and then run it on different platforms. It felt like a good dream to share: it wasn't pretty to rewrite the same stuff over and over. If your company wanted to support Windows and Apple, then you needed two separate teams writing the same application for two separate OSs. That felt like a huge waste.

We tried hard. We looked at every C compiler flaunting cross-compilation, every database driver promising vendor independency, every high-level approach touting push-button code generation. The more these solutions became sophisticated, the less they seemed to work.

We blamed Microsoft, Oracle and other corporate lockers-in for that sorry state of affairs. We were wrong. As it turned out, we were just trying to solve the wrong problem.

The real problem was not a technological issue: it was a usability issue, a culture issue, and a marketing issue. Different platforms approach the same domains differently, and their relative value lies in those differences, not in the common denominator. At one point, Java managed to solve the technological problem for good, and that was the point where we realized the awful truth: cross-platform compatibility was not important. It never had been.

So we quit trying to solve that problem. Instead, we left it behind by moving up a level and inventing a new, shared platform on the Web. (I still see companies pursuing push-button tools that generate or translate code for the CLR and JVM alike. That saddens me: somebody is still working on the wrong problem.)

Now, as it happens in IT, we're running another iteration of facing the same issues and making the same mistakes. We have multiple devices (PCs, smartphones, tablets), so we'd like to use the same software all over the spectrum. That's where Windows 8 seems to be going: you have the same OS on your tablet and your PC, so you can leverage the same technologies on both. And once again, this isn't going to work, because a tablet and a PC are different, and all those subtle and not-so-subtle differences pile up to require different approaches. Convergence is not important, interoperability is. Broad commonalities are not important, tiny details are. And please, Microsoft, get over it: the OS is not important, the user experience is.

That's why I think that Windows' current approach to tablets and smartphones is fundamentally, tragically, so-fucking-broken-it-cannot-be-fixed wrong.

Tuesday, August 9, 2011

Get Your Ruby Project on Travis and Have a Martini in 15 Minutes

Almost overnight, every Ruby project out there seem to be moving to Travis. Travis is a dead-simple, community-owned build system. I'm usually too lazy to put all my projects on automated build. Travis took away my excuses by getting me from zero to the first build in a matter of minutes.

Here are step-by-step instructions to get your project on Travis and have a delicious Martini Cocktail in about 15 minutes. Please note that preparing the Martini will take about 5 minutes, so the Travis part should take just 10 minutes of your life.

Check That Your Project Has What It Takes (2 minutes)

Your project needs three prerequisites to get on Travis:
  1. It's a Ruby project on GitHub.
  2. It uses Bundler to manage its gems. (Actually, that’s not strictly necessary, but it will make it easier to set up Travis.)
  3. You can run the project's tests with a single command. A Rake task is typical, but other commands (like, say, bundle exec rspec spec) are also fine.
Ultimately, you should be able to test your project on a new machine by just doing a bundle install followed by the test command. If your setup is more complicated, then you'll need extra work to put the project on Travis. It's probably a good idea to make your project very easy to setup, whether or not you want to use Travis.

I’ll assume that your project meets the three prerequisites, and that you can run your test with bundle exec rake test.

Create a Travis Configuration File (3 minutes)

Commit a new file named .travis.yml in your project root. Here is what mine looks like:


All the entries have sensible defaults, so your configuration could be even simpler. For example, if you skip the script property, then Travis will try bundle exec rake, or just rake if you're not using Bundler. You can find more details on the Travis configuration page.

Activate Your Project on Travis (2 minutes)

Go to http://travis-ci.org and sign in with your GitHub account. Grant Travis read/write access to your GitHub. You should see your private Travis build token on your profile page:


You don't really need to care about the token now - but while you’re on the profile page, flip the switch for the project that you want to build with Travis.

Run Your First Build (3 minutes)

Push to your git repository (make sure that you committed the .travis.yml file), then go to the Travis home page and sit back as Travis adds your project to its build queue, installs the bundle and runs the tests. When the build is done, check your email to find a little love message from Travis - and congratulations for getting it green (or red)!

If You're Curious...

Why did Travis require write access to your GitHub account? That's because Travis automagically configures GitHub to be notified when you push to the project. Go check it if you like: on your project's admin page on GitHub, follow Service Hooks, and click on the Travis hook. The configuration should look like this:


If you click Test Hook, Travis should schedule a build right now.

Prepare the Martini (5 minutes)

Fill a frozen cocktail glass with cold gin, add a touch of vermouth and stir. Garnish with an olive.

(If you’re in a hurry, you can merge this step with the previous one, thus sparing 3 minutes and keeping yourself busy as Travis is building your project.)

Drink the Martini (extra quality time)

I don’t think you need my help here. Just find good company and enjoy. Drink responsibly!