Entity Framework 6.0

So you want to contribute to EF?

Part 3: Testing

This is the third part of a series providing some background to those who may want make contributes to the Entity Framework. This post will cover the types of tests that at exist and expectations around writing new tests.

If you make a contribution to EF it is expected and required that you will write tests for that contribution and that all existing tests will continue to pass. You can run the tests from the Visual Studio solution and running the build command also runs all tests. Build must pass before any change can be pushed to master.

If you expand the Tests folder in the VS solution you will see projects for three kinds of test: unit tests, functional tests, and VB tests. All the tests make use of the xUnit test framework for which there are several runners available—my personal favorite is TestDriven.NET.

Unit tests

Unit tests test code at the unit level—that is at the class and method level. Some things to keep in mind when writing unit tests:

Functional tests

Functional tests exercise many units of code to ensure that all the units all work together as expected. (The exact level varies such that some EF functional tests may be more correctly called integration tests.) Examples include writing some data to the database and then querying it back to ensure that it round-trips correctly, or building and validating an entire Code First model.

Some things to keep in mind when writing functional tests:

VB tests

The VB tests test Visual Basic-specific code, mostly involving Visual Basic code generated by the Database First T4 templates. There is no expectation that VB tests be added for most changes.

Writing and naming tests

In general, each test method should be a simple, self-contained test. It is fundamental that every test be easy to read and understand and that the expected outcome is readily apparent. If a test fails then it should not require debugging to figure out how the test code failed. (It may of course require debugging to discover the root cause of the failure in the product code.)

Test method names

When reading a test method name it should be clear: For example:
[Fact]
public void Add_starts_tracking_a_new_entity_and_puts_it_in_the_Added_state()
{
    // Test some stuff...
}

Keeping tests simple and clean means that good test code does not have the same requirements as good product code. In particular:

Tests for core code

We faced a difficult situation when deciding to make EF open source—what do we do with all the tests written by both product developers and the QA team over the years? Making most of the DbContext tests and a significant proportion of the Code First test open source was not a problem and we have done that.

On the other hand, most of the test for the core code and almost all of the tests written by the QA team could not be made open source. I won't go into the reasons for this, but the upshot is that if you run “build” on the open source code most of the 6000+ tests that run are for DbContext and Code First. This isn't as bad as it might seem because the DbContext and Code First code is dependent on the core code and hence you get a lot of coverage of the core just through running these tests.

One thing this does mean is that if you make changes to the core code then you are likely to find that the code you are changing doesn't have any existing public tests. (We can still run some tests internally but as much as possible we want tests to be public.) It is expected that you write tests for this code as part of your change so that we gradually build up direct test coverage for the core code. If doing this is such a burden that you feel you can't make a contribution because of it then contact the EF team and we will try to figure out a solution.

Designing and refactoring for testability

A lot of the EF code, especially in the core, was not designed for testability. This means that you may need to do some refactoring in order to introduce seams, make classes mockable, and so on. This video has a good introduction to testable code.

All new code should be designed and implemented with testability in mind. Just because you see patterns or practices in the existing code that are not testable doesn't mean that we want the code to stay that way or that new code should follow the same patterns. Design and refactor for testability so that we can keep improving the code base.

Summary

All contributions must be unit tested and most must also have functional tests. All tests use xUnit and should be as simple and clear as possible.

If there is anything here that you need more information about don't hesitate to contact me or others on the EF team. In the next post I'll look at the developer experience.


This page is up-to-date as of July 19th, 2012. Some things change. Some things stay the same. Use your noggin.