In the previous entries in this series, you've learned about the basics of unit testing, and you've seen how to create a very basic unit test. In this post, you will learn how to fully use NUnit's Assert class to create a full suite of unit tests. This post builds off the sample described in the previous post, so be sure to check it out if you want to follow along.
Asserting Equality
In the last post, you saw one example of how to use the Assert.AreEqual method to verify that two objects are equal. As you will soon see, most methods on the Assert class have a ton of overloads. You can think of the method as having two levels of overloads: the first for all the various types you could pass in (it has specific overloads for most primitive types as well as more generic versions that work with anything that derives from object), and the second level for controling the message that is shown when the Assert fails. The type-based overloads are self-explanatory and are handled by the compiler for you automatically, so we'll ignore those and focus on the overloads that look like Assert.AreEqual(expected,actual,message) and Assert.AreEqual(expected,actual,message,params). Let's incorporate one of these overloads into our unit test from the last post. Here's the original, unmodified test:
1: /// <summary>
2: /// Verifies that the balance increases
3: /// by the appropriate amount.
4: /// </summary>
5: [Test]
6: public void Deposit_AddsValueToBalance()
7: {
8: Account account = new Account();
9: //account.Balance is currently zero.
10: account.Deposit(100);
11:
12: Assert.AreEqual(100, account.Balance);
13: }
Let's change our assumption about what the Deposit method should be doing. Let's say that this is an awesome bank that automatically adds a 10% match to anything that you deposit. We could just change the first parameter to Assert.AreEqual to 110, but what happens if we run that test? All you will see is the 'expected (110), actual (100)' message. It doesn't tell you much about why 110 was expected. That's where the third parameter comes in handy:
1: /// <summary>
2: /// Verifies that the balance increases
3: /// by the appropriate amount.
4: /// </summary>
5: [Test]
6: public void Deposit_AddsValueToBalance()
7: {
8: Account account = new Account();
9: //account.Balance is currently zero.
10: account.Deposit(100);
11:
12: Assert.AreEqual(110, account.Balance, "Deposit bonus was not applied!");
13: }
Now when you run the test, you will see the helpful message that was supplied as the third parameter to AreEqual. The fourth parameter, a params array of objects, behaves like the String.Format method: the third parameter becomes a format string, and the fourth parameter is the set of values to insert into the format string. This could be useful for logging additional information about the unit test failure.
For both double and float types, the AreEqual method has an additional set of overloads that look like Assert.AreEqual(expected,actual,tolerance). The tolerance parameter allows you to tell NUnit how close two floating-point values have to be in order to be considered equal. Going back to our Deposit_AddsValueToBalance test, what if the bonus wasn't *quite* 10%, but was more like 9.75989%? Well, we could calculate exactly what the bonus would be and pass that in as the expected value, then apply it, or we could specify a tolerance of 0.01 and leave the expected value as 110, like so:
1: /// <summary>
2: /// Verifies that the balance increases
3: /// by the appropriate amount.
4: /// </summary>
5: [Test]
6: public void Deposit_AddsValueToBalance()
7: {
8: Account account = new Account();
9: //account.Balance is currently zero.
10: account.Deposit(100);
11:
12: Assert.AreEqual(110, account.Balance, 0.01, "Deposit bonus was not applied!");
13: }
In addition to primitive types, NUnit has some "special" support for Arrays and Collections. Typically in the .NET world, Equality is determined by an object's Equals method. Derived types are responsible for overriding that method if they wish to define equality as anything other than the default behavior inherited from the object class. NUnit fudges this definition a bit for Collections and Arrays: two collections (or arrays) are considered equal if they have the same number of items and all their corresponding elements are equal.
There is a corresponding inverse method to AreEqual called (not surprisingly) AreNotEqual. The obvious difference is that AreNotEqual verifies that two objects are different from one another. AreNotEqual has the same set of overloads as its complementary method.
Asserting Sameness
Next up is the AreSame method. You may be wondering "what's the difference between 'same' and 'equal'?". Objects A and B are equal if A.Equals(B) returns true. Remember that by default all objects inherit an Equals method from the base object class, and that derived classes can implement custom equality checks as needed, so the exact definition of equal depends on what you are comparing. 'Same' is much simpler: objects A and B are the same if they point to the exact same object in memory.
The difference between 'equal' and 'same' may sound a bit confusing if you aren't comfortable with the concept of pointers and object references, so check the documentation here if you are still unclear.
To demonstrate this difference, let's create a new method that looks up account information and verify that repeated calls to the method return the same account information instance. NOTE: I am intentionally not doing things in a test-driven manner right now. I don't want to muddy the waters with trying to explain that concept at the same time I'm explaining the asserts. A proper treatment of test-driven development is coming Real Soon(tm)!
First, let's add some new properties to our Account class along with an overloaded Equals method:
1: /// <summary>
2: /// A bank account.
3: /// </summary>
4: public class Account
5: {
6: #region Public Properties
7:
8: /// <summary>
9: /// The ID of the account.
10: /// </summary>
11: public int AccountID { get; private set; }
12:
13: /// <summary>
14: /// The name of the account owner.
15: /// </summary>
16: public string Owner { get; private set; }
17:
18: /// <summary>
19: /// The current account balance.
20: /// </summary>
21: public float Balance { get; private set; }
22:
23: #endregion
24:
25: #region Public Methods
26:
27: ----Snip----
28:
29: /// <summary>
30: /// Compares the current object to the specified object.
31: /// </summary>
32: /// <param name="obj"></param>
33: /// <returns>True if the accounts have the same AccountID,
34: /// false otherwise.</returns>
35: public override bool Equals(object obj)
36: {
37: Account account = obj as Account;
38:
39: if (account == null)
40: {
41: return false;
42: }
43: else
44: {
45: return AccountID == account.AccountID;
46: }
47: }
48:
49: /// <summary>
50: /// When you override Equals, you have to override
51: /// GetHashCode, too...
52: /// </summary>
53: /// <returns></returns>
54: public override int GetHashCode()
55: {
56: return AccountID.GetHashCode();
57: }
58:
59: #endregion
60: }
Next, let's add a method to look up and return a 'dummy' Account object on demand:
1: /// <summary>
2: /// Gets the specified account.
3: /// </summary>
4: /// <param name="accountID"></param>
5: /// <returns></returns>
6: public static Account Lookup(int accountID)
7: {
8: return new Account {AccountID = accountID, Owner = "John Doe"};
9: }
Finally, let's create our test:
1: /// <summary>
2: /// The Lookup should return the exact same instance
3: /// for all lookups on a specific ID.
4: /// </summary>
5: [Test]
6: public void Lookup_ReturnsSameInstance()
7: {
8: Account account = Account.Lookup(1);
9:
10: Assert.AreSame(account, Account.Lookup(1));
11: }
Go ahead and build the project and run the test. What happened? The test failed because even though we are returning an identical object for both calls, we aren't actually returning the same object. Let's correct that by storing some static dummy Account instances; we'll return one of these instead of creating a new instance from now on:
1: /// <summary>
2: /// These are our dummy accounts.
3: /// </summary>
4: private static Account[] mAccounts = new Account[]
5: {
6: new Account{ AccountID = 1, Owner = "John Doe" },
7: new Account{ AccountID = 2, Owner = "Jane Doe" }
8: };
9:
10: /// <summary>
11: /// Gets the specified account.
12: /// </summary>
13: /// <param name="accountID"></param>
14: /// <returns></returns>
15: public static Account Lookup(int accountID)
16: {
17: //Arrays are 0-based, accountIDs are 1-based, so we shift them.
18: return mAccounts[accountID - 1];
19: }
Build and re-run the test, and you should get a success message.
Similar to the AreEqual method, the AreSame method has a logical inverse: the AreNotSame method.
Asserting Greatness
NUnit includes four methods (with overloads) for asserting various inequalities: Greater, GreaterOrEqual, Less, and LessOrEqual. The intent of these methods should be obvious, but they differ in one major one from the other assertions we've seen so far. Recall that the basic versions of both AreEqual and AreSame took an expected argument first and the actual value second. Applying that same logic, you might expect that expressing the inequality x is greater than y would look like Assert.Greater(y,x), but it's actually the opposite. I can't tell you how many times I've seen this inconsistency bite developers; it doesn't help that the parameters have less-than-helpful names, like arg1 and arg2. I don't know why they couldn't have used something more obvious and intuitive, like maybe left and right...
Enough complaining, Let's write some code! Let's add a new deposit to our account class called RandomDeposit. This method is very different from our standard Deposit method. Instead of depositing the specified amount, RandomDeposit will deposit a random amount that is anywhere from 0.5 to 1.5 times the specified amount. The method looks like so:
1: /// <summary>
2: /// Deposits a random amount that is between 0.5
3: /// and 1.5 times the specified amount.
4: /// </summary>
5: /// <param name="amount"></param>
6: public void RandomDeposit(float amount)
7: {
8: Random rand = new Random();
9: double multiplier = rand.NextDouble();
10:
11: Balance += (float)(amount*(0.5 + multiplier));
12: }
Because of the randomness in the method, it's going to be very hard to write a unit test using the Assert.AreEqual method. Instead, we'll use the GreaterOrEqual method to assert that the deposited amount is at least 0.5 times the amount we deposited. Here's the unit test:
1: /// <summary>
2: /// The method should deposit between 0.5 and 1.5
3: /// times the specified amount.
4: /// </summary>
5: [Test]
6: public void RandomDeposit_DepositsExpectedAmount()
7: {
8: Account account = new Account();
9:
10: account.RandomDeposit(100);
11: Assert.GreaterOrEqual(account.Balance, 50);
12: Assert.LessOrEqual(account.Balance, 150);
13: }
If you aren't already building and testing by habit, go ahead and build the project and run the new test.
Asserting Typeiness (If Steven Colbert can do it, so can I!)
The Assert class includes methods for asserting things about the type of an instance. You can check whether or not an object is of a given type using the IsInstanceOfType method. The first parameter is the expected type of the object, the second parameter is the actual object. Let's make our Account class implement the ICloneable interface, then write a test to verify that our clone actually is of type Account:
1: /// <summary>
2: /// A bank account.
3: /// </summary>
4: public class Account : ICloneable
5: {
6: ----SNIP----
7: /// <summary>
8: /// Clones the current account.
9: /// </summary>
10: /// <returns></returns>
11: public object Clone()
12: {
13: return new Account {AccountID = AccountID, Balance = Balance, Owner = Owner};
14: }
15: ----SNIP----
16: }
Here's the corresponding test case:
1: /// <summary>
2: /// Verifies that a complete clone of the account
3: /// is returned.
4: /// </summary>
5: [Test]
6: public void Clone_ReturnsAccountClone()
7: {
8: Account account = Account.Lookup(1);
9:
10: object clone = account.Clone();
11:
12: Assert.IsInstanceOfType(typeof (Account), clone);
13: }
The IsInstanceOfType and IsAssignableFrom methods are very similar. Under the covers, they're just calling the corresponding members of the System.Type class. Check the documentation on MSDN if you are curious about the subtle differences between the two methods, but for the most part, you can use them interchangeably.
As with most assert methods, there are various overloads of both IsAssignableFrom and IsInstanceOfType. Each also has a set of complementary Not methods: IsNotAssignableFrom and IsNotInstanceOfType.
Asserting Nothingness
Sometimes the right result is a null result. Let's look again at our Account.Lookup method. Right now, we're not really handling the case of an account ID that doesn't exist. Let's modify the code so that it returns null when given an invalid account ID:
1: /// <summary>
2: /// Gets the specified account.
3: /// </summary>
4: /// <param name="accountID"></param>
5: /// <returns></returns>
6: public static Account Lookup(int accountID)
7: {
8: if (accountID < 1 || accountID > mAccounts.Length)
9: {
10: return null;
11: }
12:
13: //Arrays are 0-based, accountIDs are 1-based, so we shift them.
14: return mAccounts[accountID - 1];
15: }
And let's write a new test case to verify this:
1: /// <summary>
2: /// The lookup should return null when given an ID
3: /// that doesn't correspond to an account.
4: /// </summary>
5: [Test]
6: public void Lookup_ReturnsNullForInvalidId()
7: {
8: Assert.IsNull(Account.Lookup(0));
9:
10: Assert.IsNotNull(Account.Lookup(2));
11: }
Here we've used the Assert.IsNull method. This is a very simple assert: it simply checks that the parameter is null. Like everything else, it has a complementary method that will test that something is not null.
Asserting Truthiness (or Falsiness)
Everything we've asserted so far could actually be expressed using one of the most basic assertions: IsTrue. This method asserts that a boolean input is true. It has a complementary IsFalse method that can be used to assert that an input is false. These methods can be used to test anything that you can express as a boolean condition. Let's rewrite our previous Lookup test using only IsTrue instead of IsNull to see this:
1: /// <summary>
2: /// The lookup should return null when given an ID
3: /// that doesn't correspond to an account.
4: /// </summary>
5: [Test]
6: public void Lookup_ReturnsNullForInvalidId()
7: {
8: Assert.IsTrue(Account.Lookup(0) == null);
9: //Assert.IsNull(Account.Lookup(0));
10:
11: Assert.IsTrue(Account.Lookup(2) != null);
12: //Assert.IsNotNull(Account.Lookup(2));
13: }
The test should produce identical output because it is logically equivalent to the original. You might be tempted to just say "forget about all these other asserts, I'll just use IsTrue for everything!", but that's a terrible idea. The various other assertions give you a lot more information when something goes wrong than IsTrue will. For example, if GreaterThan fails, it will tell you the values of both parameters. If you expressed the test using only IsTrue, you would get a very unhelpful message that says "Expected: True, Actual: False". Sure, you can probably work backwards, add some logging, etc, to figure out what's going on, but why not use the more powerful GreaterThan method to begin with?
Asserting Failuriness
Sometimes you just want a test to fail. Maybe the test isn't finished, or the test couldn't perform some setup correctly, or maybe you need to test for something that is beyond what the built-in assertion methods can handle. Assert.Fail to the rescue! Calling this method will instantly fail a test (assuming you haven't done anything silly like wrapped the call with a try-catch block, which we will look at in a future post).
Asserting Exceptioniness
We've tested that things work so far, but how do we test that things explode? Right now, there's nothing in our Deposit method that prevents us from depositing negative amounts. Let's add some logic to throw an exception:
1: /// <summary>
2: /// Deposits the specified amount.
3: /// </summary>
4: /// <param name="amount"></param>
5: public void Deposit(float amount)
6: {
7: if (amount <= 0)
8: {
9: throw new ArgumentOutOfRangeException("amount", amount, "Must be greater than zero.");
10: }
11:
12: Balance += amount;
13: }
Error-handling code is good, but it still needs to be tested. NUnit has the ExpectedException attribute that you can use to verify that an exception is thrown, but I hate this attribute. What the attribute is really doing is verifying that something somewhere in your test case is throwing an exception, not that the exception is actually coming from where you want it to come from. Instead, I prefer to go with this model:
1: /// <summary>
2: /// The method should throw an ArgumentOutOfRangeException
3: /// if you pass in a negative value.
4: /// </summary>
5: [Test]
6: public void Deposit_ThrowsExceptionOnNegativeAmount()
7: {
8: Account account = new Account();
9:
10: try
11: {
12: account.Deposit(-100);
13: //The following line will only be executed if the Deposit method
14: //failed to throw an exception.
15: Assert.Fail("Expected ArgumentOutOfRangeException was not thrown!");
16: }
17: catch (ArgumentOutOfRangeException)
18: {
19: //Ok, this is an expected exception.
20: }
21: }
It requires a bit more code, but this version verifies that the correct type of exception is thrown in exactly the right spot.
Other Ways to Assert RoXXorness
The methods we've looked at so far are just the ones that I have found myself using often over the last several years. NUnit includes other methods that you can use to assert various things about your objects, including:
- Assert.Contains - Given an object and a list, this method asserts that the collection contains the specified object.
- Assert.IsEmpty - Given a collection (or a string), asserts that the object contains no items.
- Assert.IsNaN - Both double.NaN represent the 'not-a-number' condition (often caused by division by zero). You can test for this condition using the IsNaN assert.
There is even more...
Recent versions of NUnit have added additional utilities, asserts, etc. to simplify your testing. You can find out more about them here. We might look at those in a future post, but I really don't find myself using most of them in my day-to-day testing, and I think most developers can get by just fine without them.
In the next post in this series, we'll look at some more complicated testing scenarios as well as common testing problems and strategies for overcoming them.
Tags: