To kill a mockingbird

March 15, 2009 12:09
There is another update for Mocking Frameworks Compare that allows you to evaluate the performance of the frameworks. The same standard mocking operations being called in a sophisticated loop[1]. Nothing fancy.

 For now, don't expect any comments on whether this comparison is relevant or not, whether it's important or not, and what could you do with it. Only numbers.

But they are nice.

Mocking methods.
Moq      : 100 repeats:  77,281 +- 6%    msec
Rhino    : 100 repeats:  98,784 +- 2%    msec
NMock2   : 100 repeats:  53,934 +- 6%    msec
Isolator : 100 repeats: 312,595 +- 1%    msec

Mocking events.
Moq      : 100 repeats: 179,061 +- 3%    msec
Rhino    : 100 repeats: 159,066 +- 2%    msec
NMock2   : 100 repeats:  61,919 +- 3%    msec
Isolator : 100 repeats: 242,173 +- 2%    msec

Mocking properties.
Moq      : 100 repeats: 169,055 +- 2%    msec
Rhino    : 100 repeats: 127,463 +- 10%   msec
NMock2   : 100 repeats:  25,869 +- 4%    msec
Isolator : 100 repeats: 287,091 +- 3%    msec

Mocking method arguments.
Moq      : 100 repeats: 288,994 +- 2%    msec
Rhino    : 100 repeats: 118,773 +- 2%    msec
NMock2   : 100 repeats:  52,253 +- 1%    msec
Isolator : 100 repeats: 343,406 +- 15%   msec

Partial mocks.
Moq      : 100 repeats: 255,542 +- 2%    msec
Rhino    : 100 repeats: 148,065 +- 4%    msec
Isolator : 100 repeats: 473,029 +- 29%   msec

Recursive mocks.
Moq      : 100 repeats: 556,984 +- 8%    msec
Rhino    : 100 repeats: 554,193 +- 9%    msec
Isolator : 100 repeats: 657,907 +- 35%   msec

Those results were obtained on a pretty standard Vista laptop, during a quiet time. Your mileage may vary.

 
Footnotes.
  1. Measuring part, courtesy of a performance raconteur Vance Morrison, is based on Code Timers idea he implemented yonks ago. It's great as it incorporates the basic statistics, all those mean/median/standard deviation things that give you a much richer understanding of a measurement. Feel free to reuse in your own benchmarking! 

Tags: , , , , ,
Actions: E-mail | Permalink | Comments (3) | Comment RSSRSS comment feed

Comments

Comments are closed