Unit testing is a software development methodology where the developer writes tests that validate a small portion of code. A unit test is usually composed of three ingredients: (1) data such as strings or integers, (2) a sequence of method calls that exercise the code under test and (3) assertions that verify the behaviour.
int AddBroken(int x, int y) {
if (x > 1000) x = 1000; // it’s too big!
return x + y;
}
[TestMethod] public void AddOneTwo () {
int z = AddBroken(1, 2);
Assert.AreEqual(3, z);
} // this test passes!
In many cases, the data provided by the developer is irrelevant, partial or redundant. Pex is an automated test generation tool for .NET that automatically generates the data for your unit tests.
Given a parameterized unit test, i.e. a unit test method with parameters, Pex tries to generate data inputs to cover all reachable branches. To achieve this, Pex analyses the code as it is running and uses the observed program behavior to systematically generate an exhaustive set of data inputs that will cover many program branches. Since every assertion in the code is actually a conditional branch, Pex also finds bugs by trying to determine inputs that will cause assertion to fail, in the code or in the tests. In the end, Pex produces a small test suite that achieves high code coverage.
// parameterized unit test written by hand
[PexMethod] public void AddAnyNumbers(int x, int y) {
// this test should pass for any x and y
int z = AddBroken(x, y);
Assert.AreEqual(x + y, z);
}
// automatically generated unit test by Pex
[TestMethod] public void AddAnyNumbers01() {
AddAnyNumbers(1001, 0);
} // this test fails the assert 1001 + 0 != 1000!
Unit tests should be executable in isolation from the environment. However, in practice, many unit tests are actually integration tests as they communicate with a database or rely on the file system or even the current machine time. Such unit tests are usually slow, non-deterministic, hard to setup and diagnose. To deal with these issues, developers should introduce abstraction layers, i.e. interfaces, between the code and the environment. To greatly simplify this task, Pex provides a framework, Stubs, which automatically generates implementations of the interfaces. In many other cases, environment dependencies are hard-coded, without using interfaces. For those scenarios, if it is not feasible introduce abstraction layers, Pex provides a framework, Moles, that allows replacing any .NET method with a user-defined delegate, in effect detouring calls to environment-facing methods at test time. This effectively isolates unit tests without changing the application code, enabling testing of SharePoint applications, ASP.NET, etc.
int AddBroken(int x, int y) {
if (DateTime.Now == new DateTime(2000,1,1)) throw new Exception(“y2k bug!”);
return x + y;
}
// parameterized unit test written by hand
[PexMethod] public void AddAnyNumbers(int x, int y, DateTime now) {
// this test should pass for any x, y and current time ‘now’
MDateTime.NowGet = () => { return now; };
// let’s make DateTime.Now return ‘now’ always
int z = AddBroken(x, y);
Assert.AreEqual(x + y, z);
}
Pex is an incubation project for Visual Studio, developed by Microsoft Research, and currently available via http://msdn.microsoft.com/devlabs.
A guest post by: Peli de Halleux, Senior Research Software Developer Engineer, Microsoft Research, Redmond.