Introduction to Pex Stubs

July 5, 2009 20:17

This post is a walk-through for using Stubs, an interesting approach to mocking from Microsoft Research[1].

Typically, while mocking something you set up expectations via expression trees or string based APIs. The framework would then, at runtime, dynamically generate the code you need.

Dynamically.

Stubs guys took a different route: they generate the code statically. Right click on the method you want to test, click a button and a separate project with the stubs you need gets created. Just go and use them.

Let's look at a trite example.

public class Movie
{
    public string Name { get; set; }
    public string Genre { get; set; }
}
 
public interface IMovieProvider
{
    List<Movie> ListMovies();
}

Imagine we have a class and an interface as defined above. Below is the client - it gets the list of movies, filters them by the given genre and returns the names only:

public class MoviePage
{
    private readonly IMovieProvider _provider;
 
    public MoviePage(IMovieProvider provider)
    {
        _provider = provider;
    }
 
    public List<string> ListNamesByGenre(string genre)
    {
        var movies = new List<Movie>();
        foreach (var movie in _provider.ListMovies()) {
            if (movie.Genre == genre)
                movies.Add(movie);
        }
 
        var names = new List<string>(movies.Count);
        foreach (var movie in movies) {
            names.Add(movie.Name);    
        }
        return names;
    }
}

Note that we use Dependency Injection pattern and pass the provider in the constructor - thus we'd be able to pass the mocked provider in our test[2]. 

Getting started with Pex Stubs.

Well, it's easy. You install Pex[3], kick off Visual Studio, right click on the method and generate a helper project that will contain the stubs.



A small wizard would pop up. Note that it needs to know your
unit test framework and has Visual Studio Unit Test preselected - so change that if you're using a different one (for now, VS Unit Test, NUnit, MbUnit and xUnit.net are supported).

Finally we write a test. To placate those shrill voices saying
that the whole ListNamesByGenre method (10 lines of code) can be written in one line using Linq, let's do it in the test. Well, unit tests should be as brief as possible so that they're understandable and maintainable, right?

[TestFixture]
public class MovieTests
{
    [Test]
    public void ListMovieNamesByGenre()
    {
        //Arrange
        var movies = new List<Movie> {
            new Movie { Genre = "cartoon", Name = "Ice Age 3" },
            new Movie { Genre = "action", Name = "Star Wars" },
            new Movie { Genre = "action", Name = "The Matrix" },
        };
        var provider = new SIMovieProvider {   //our stub
            ListMovies = () => { return movies; }
        };
        
        //Act
        const string genre = "action";
        var page = new MoviePage(provider);
        var movieNames = page.ListNamesByGenre(genre);
 
        //Assert
        var names = movies.FindAll(m => m.Genre == genre).ConvertAll(m => m.Name);
        Assert.That(movieNames, Has.Count(names.Count));
        Assert.That(movieNames, Is.EquivalentTo(names));
    }
}

Delegates approach.

Stubs framework is built on delegates. The stubs you generate end up in one huge file, where every stub name is just a corresponding real class or interface name prefixed with "S" (in the example above we're using SIMovieProvider). Every mockable method is either a Func or an Action.

In our case, autogenerated code looks like this (simplified version):

public partial class SIMovieProvider : Microsoft.Stubs.Framework.StubBase, IMovieProvider
{
    public Func<List<Movie>> ListMovies;
 
    List<Movie> IMovieProvider.ListMovies()
    {
        var stub = this.ListMovies;   //the stub we define in a test
        if (stub != null) {
            return stub.Invoke();
        }
        else {
            var defaultBehavior = base.FallbackBehavior;
            return defaultBehavior.Result(this);
        }
    }
}

The approach is beautiful in its simplicity. If you've defined a stub in the test, your code gets called - otherwise it falls back to the default behavior of the stub. And obviously you can debug the test and step into and see your stub code being executed.



Conclusion.

Among drawbacks, one can mention that you need to regenerate the stubs every time you change the original code - to feed your unit tests with up-to-date mocks (however, probably it's all about creating hooks in a build process, for an automatic update).

Another thing - the "autogenerating stuff" origin of Stubs has made it.. um, less readable for demanding users of other mocking frameworks. Say, if instead of stubbing an interface IMovieProvier we take a class MovieProvier, the method name to stub becomes ListMovies01 because otherwise you'd have a build error "The type already contains a definition for ListMovies". (Perhaps a somewhat better approach to overcome that would be an inner class containing the overrides with the exact names.)

Other than that, Stubs is very interesting. It is a framework without
an API, so mocking is one virtual call away (say, mocking a method with Stubs is 1000 times (!) faster than with Isolator[4]). The limitations are more or less the same as in other free mocking frameworks: it cannot generate stubs for static and non-virtual methods and sealed classes. But it's powerful enough to support partial and recursive mocks. 

And it's only version 0.14. Looking forward to the times of Stubs 1.0
!  



Footnotes.

  1. Stubs framework (read more in this pdf) is a part of Pex project, a tool to autogenerate unit tests with 100% code coverage - but that's beyond the scope of the post. Find more about Pex on their official page.
  2. Often, you may come across some (legacy?) code that uses static classes like MovieProviderFactory.GetProvider() to get the provider directly inside the method. It's usually a good idea to write a test that is not dependent on the provider: otherwise the test can get bound to a database or file system, or fail if something is wrong with the provider itself. All that is fatal for a good unit test because it makes the test either brittle or slow or confusing. So, if a method you want to test is bound to static providers... you have two options:
    • Refactor it. Either by passing the provider as a parameter, as we do in the example, or refactor the factory to be substitutable with your own mocked one.
    • Use Typemock Isolator. It's the only mocking framework for .NET that is not free but it's the most powerful one. Among other things, it can mock static methods.
  3. Pex download page is available here. This article is written using the academic edition, version 0.14.40610.2 published on 11 June 2009.
  4. Check out perf comparison in Mocking Frameworks Compare.


Comments

Comments are closed