In my current project we have reached a very good code coverage percent (98,7 %, yes we are proud) - but we are aspiring to take it a step further.
We are now constructing a series of integration test, used to run through the actual production code an verifying that everything works as expected.
Said and done - I implemented a few test that did that. Soon though some questions and problems arose;
- Integration test assumes that something is released and the that tests are executed against that release
- You don’t want the integration test to be executed when a developer is running the unit test on his development machine inside Visual Studio.
When looking around on the net it seems that many people are missing the Category-attribute from NUnit in MSTest. That looked like a very nice way to solve the problem but it is not present in MSTest. They instead talk about test-lists but that doesn’t solve the problem with running the unit test locally on the developer machine.
We solved it with a bit more brute force… I created a separate solution with just the integration tests in it. That solution is added to the SolutionToBuild property group in my build script so that it is compiled. Then I run the test in the integration test dll via an exec-task that run MSTest. That part is only run when a new deploy is made.
Quite easy and maybe not the most beautiful solution … but it works.
[UPDATE]