There is no shortage of Test/Behavior-Driven Development frameworks for .NET. You have SpecFlow, Machine.Specifications (MSpec), and even Cucumber. While these frameworks have been around for a while, I felt they weren’t right for me for various reasons, so I created SpecsFor. SpecsFor is a TDD/BDD framework that has been naturally cultivated from use on two projects, including one commercial app. The first release of SpecsFor is in the works, but first, I need feedback on some changes I’m considering to the core API.
But why??
I have previously shown the SpecsFor test helpers that are used on RageFeed (the never-to-be-released social networking app I’m working on). My goal with SpecsFor was to completely eliminate testing friction by creating a simple set of conventions and templates to enable rapid spec creation, and I feel that SpecsFor has actually come very close to realizing my zero-friction vision. A variant of SpecsFor is also used at my day job, and I have recently found myself copy-pasting the core class into other projects that I work on. This is not conducive to maintaining SpecsFor, and I feel it’s time it grew up and became a proper library.
What is it?
SpecsFor is primarily a base test fixture that establishes a clean way of performing simple Behavior-Driven Development style tests while providing an auto-mocking container and other utilities to reduce common sources of testing friction. It greatly simplifies common operations such as initializing the class under test, establishing context, and creating mock objects. When combined with a library like Should, you can quickly create very readable specifications.
SpecsFor exposes overridable template methods corresponding to the various steps of the BDD process: to establish context (the Given), to perform an action (the When), and to verify outcomes (the Then). For now, the official version is built on NUnit, StructureMap, and Moq, but it will be available for the IoC library and testing framework of your choice soon (I have already created versions of SpecsFor that operate on MS Test and Ninject).
SpecsFor also includes a small set of Resharper templates. These templates make it easy to create new specs and test cases.
SpecsFor Today
The main version of SpecsFor, as seen in RageFeed, works well with the testcase class per feature pattern. Since it’s not uncommon to want to reuse some test context across multiple fixtures, the Given logic is conventionally contained in abstract base classes. The test fixtures themselves then inherit from the appropriate context, perform an action, and assert the results. Here’s an example:
public class when_the_key_is_turned : given.the_car_is_not_running
{
protected override void When()
{
SUT.TurnKey();
}
[Test]
public void then_it_starts_the_engine()
{
GetMockFor<IEngine>()
.Verify(e => e.Start());
}
}
public static class given
{
public abstract class the_car_is_not_running : SpecsFor<Car>
{
protected override void Given()
{
base.Given();
GetMockFor<IEngine>()
.Setup(e => e.Start())
.Callback(() => Console.WriteLine("VRRRROOOOOOOM"));
}
}
}
While this works relatively well for most simple cases, it’s not without flaws. It clearly violates the principle of “favor composition over inheritance.” It often leads to fixtures that identical except for their base classes. It is also difficult to combine contexts. Indeed the only option for reusing and combining contexts is to use inheritance, like so:
public abstract class the_car_is_not_running : SpecsFor<Car>
{
protected override void Given()
{
base.Given();
GetMockFor<IEngine>()
.Setup(e => e.Start())
.Callback(() => Console.WriteLine("VRRRROOOOOOOM"));
}
}
public abstract class the_car_is_parked : SpecsFor<Car>
{
protected override void Given()
{
base.Given();
GetMockFor<ITransmission>()
.SetupProperty(t => t.Gear, "Park");
}
}
public abstract class the_car_is_parked_and_not_running : the_car_is_not_running
{
//DUPLICATED CODE! YUCK!
protected override void Given()
{
base.Given();
GetMockFor<ITransmission>()
.SetupProperty(t => t.Gear, "Park");
}
}
Even this breaks down in cases where you need context from more than one chain of inheritance. Unfortunately, complex inheritance relationships can be just as detrimental to maintaining test suites as to maintaining production code.
Towards A Better API
The first official release of SpecsFor will feature an improved model for specifying test cases. I have several goals for the revised API in mind:
- Context is a distinct entity and is established without using inheritance.
-
Context can be easily mixed and matched.
-
Attaching context to a test fixture should require minimal code noise (code that isn’t testing something).
-
A single fixture can be re-used with multiple contexts.
The first three goals are “must haves” in my opinion. The last is just a “nice to have,” but I hope that I can find an approach that will facilitate better reuse. Right now, I have designed several candidate APIs that meet these goals with varying degrees of success. All assume that context setup (the Given phase of BDD) is handled by a class separate from the test fixture itself. Here are the contexts used in the example fixtures shown below:
public class TheCarIsRunning : IContext<Car>
{
public void Initialize(ITestState<Car> state)
{
state.GetMockFor<IEngine>()
.Setup(e => e.Start())
.Throws(new InvalidOperationException());
}
}
public class TheCarIsParked : IContext<Car>
{
public void Initialize(ITestState<Car> state)
{
//TODO: Setup a mock transmission for the car.
}
}
Feedback Needed
Before I make a decision on which API to proceed forward with for the first public release of SpecsFor, I’d like to get some feedback from the .NET community. Which of the candidate APIs do people prefer? Are there other options I haven’t considered? Pros or cons to one of the approaches that I overlooked? Please take a few minutes, evaluate these samples, and let me know what you think.
Option 1 – Manually specify context
The first option is the easiest to implement as it’s not very far from how SpecsFor works today. Instead of using base classes to establish the context, you would just specify which context class (or classes) to apply prior to test execution.
public class when_the_key_is_turned : SpecsFor<Car>
{
private InvalidOperationException _exception;
protected override void Given()
{
Given<TheCarIsRunning>();
//Context could be combined by making multiple calls to Given<TContext>().
}
protected override void When()
{
_exception = Assert.Throws<InvalidOperationException>(() => SUT.TurnKey());
}
[Test]
public void then_it_should_throw_exception()
{
_exception.ShouldNotBeNull();
}
}
Pros: Simple to implement. Very flexible.
Cons: Doesn’t read very well. Adds some code bloat to the spec class.
Option 2 – Attributes specify context
Instead of writing imperative code to establish the context for a spec, Option 2 would enable you to declare context by leveraging attributes. Multiple Given attributes could be applied to a spec, which would trigger the spec to be re-run multiple times, once for each attribute (and therefore once for each context or combination there of). Multiple contexts could be specified in a single Given attribute if you wanted to compose multiple contexts together.
[Given(typeof(TheCarIsRunning))]
[Given(typeof(TheCarIsParked), typeof(TheCarIsRunning))]
public class when_the_key_is_turned3 : SpecsFor<Car>
{
private InvalidOperationException _exception;
protected override void When()
{
_exception = Assert.Throws<InvalidOperationException>(() => SUT.TurnKey());
}
[Test]
public void then_it_should_throw_exception()
{
_exception.ShouldNotBeNull();
}
}
Pros: Reads well. No clutter code inside the spec; concise way of specifying context. Enables a single spec to be run multiple times with different contexts.
Cons: Run-time type validation for context types. Probably not feasible without an NUnit add-in.
Option 3 – Marker Interfaces Specify Context
Simple marker interfaces would be used to specify context in Option 3 (note that ‘Given’ is actually an interface; the ‘I’ prefix was intentionally omitted to improve the readability of the code). Contexts could be combined by implementing multiple ‘Given’ interfaces.
public class when_the_key_is_turned
: SpecsFor<Car>, Given<TheCarIsRunning>, Given<TheCarIsParked>
{
private InvalidOperationException _exception;
protected override void When()
{
_exception = Assert.Throws<InvalidOperationException>(() => SUT.TurnKey());
}
[Test]
public void then_it_should_throw_exception()
{
_exception.ShouldNotBeNull();
}
}
Pros: Reads fairly well. No clutter code inside the spec; concise way of specifying context.
Cons: Run-time type validation for context types. No way to run a single spec multiple times with different contexts.
Option 4 – Attributes Leveraging Existing NUnit Functionality
NUnit already has a way of running a test fixture multiple times with different parameters. This capability could be leveraged in order to support specifying context using attributes without an NUnit add-in, but it would require that the spec class expose a constructor as in the following example:
[Given(typeof(TheCarIsRunning))]
[Given(typeof(TheCarIsParked), typeof(TheCarIsRunning))]
public class when_the_key_is_turned4 : SpecsFor<Car>
{
private InvalidOperationException _exception;
public when_the_key_is_turned4(params Type[] context)
: base (context)
{
}
protected override void When()
{
_exception = Assert.Throws<InvalidOperationException>(() => SUT.TurnKey());
}
[Test]
public void then_it_should_throw_exception()
{
_exception.ShouldNotBeNull();
}
}
Pros: Reads well. Concise way of specifying context. Enables a single spec to be run multiple times with different contexts. Fairly simple to implement.
Cons: Spec classes must remember to implement a public constructor that passes the contexts to the base class. Run-time checking of context types.
Final Remarks
SpecsFor is meant to be a Behavior Driven Development framework that puts developer comfort and productivity first. It is built around the tools you are already accustomed to. Unlike some other BDD frameworks that focus on bridging customer requirements and unit tests through wonky (that’s a technical term) binding of English to executable tests, SpecsFor is designed to make testing easier and more productive for the developer. I would like to have the first public version out within a week, but I also want to be sure that the road SpecsFor settles on is the right road. I don’t want to make breaking changes in future releases because of short-sighted decisions today. To avoid that pitfall, I’d really appreciate your feedback. Please take a moment and leave a comment below. Let me know which options you like, which you hate, and feel free to share your own ideas about what the ideal BDD framework for .NET developers should look like. Thanks!
Tags: