Home » Development » Testing » Best all-around .NET coverage tool – OpenCover

Best all-around .NET coverage tool – OpenCover

This post is the conclusion of a series chronicling my search for a .NET coverage tool.

This is the gala awards show, where my chosen coverage tool is announced.

If you’ve come this far, you’ve probably already read the title, and it won’t surprise you to learn that I’ve chosen OpenCover. It offered the best fit for my requirements – the only areas where I found it lacking were in the “nice to haves”. Witness:

  • OpenCover is pretty easy to run from the command line – second only to NCover.
  • It can (with the help of ReportGenerator) generated coverage reports in XML and HTML.
  • OpenCover has an integrated auto-deploy, so it can be bundled with the source tree and new developers or build servers just work – dotCover has no such option, and I was not able to use NCover this way.
  • I’ve been able to link with TypeMock Isolator with little trouble, and the new Isolator may obviate the need for my small workaround.
  • It’s free. Aside from the obvious benefit, it’s nice to not have to count licenses when adding developers and/or build server nodes.
  • There’s no GUI integration, but this was a nice to have. If some developer is absolutely dying to have this, my boss’s boss has indicated that money could be available for individual licenses of something like dotCover.
  • There’s no support for integrating with IIS. We don’t need this right now, so that’s okay. Again, if we one or two developers find a need, we have the option of buying a license of some other tool. Even better, support may be coming soon.

After considering OpenCover’s strengths in the areas I absolutely needed, and its weaknesses, which all appear to be in areas that I care a little less about, I recommended it the boss’s boss, who was agreed with the assessment and was happy to keep a little money in his pocket for now.

So, I grabbed 2.0.802, incorporated it into one product’s build, and out popped coverage numbers. Very exciting. I did notice a few things, though:

  1. Branch coverage has been added since I last evaluated the product!
  2. One fairly complicated integration-style testfixture is not runnable under OpenCover – the class tested creates a background thread and starting the thread results in a System.AccessViolationException. I was unable to determine the cause of this, and have temporarily removed the test from coverage, instead executing it with NUnit directly. I’m going to continue investigating this problem.
  3. Since I’m XCopy deploying, I was bitten by the dependency on the Microsoft Visual C++ 2010 Redistributable Package – I ended up including the DLLs in my imported bundle, and all was well, but I worry a little about the stability of this solution.
  4. The time taken to execute our tests (there are over 5000, and many hit a database) increased from about 7 minutes to about 8. This is an acceptable degradation, since the test run isn’t the bottleneck in our build process.
  5. The number of “Cannot instrument as no PDB could be loaded” messages is daunting. I’m hoping that things will be improved once I get a build that contains a fix for issue 40.
Advertisements

4 Comments

  1. Shaun Wilde says:

    Hi

    Please raise the AccessViolationException issue with opencover and I’ll have a look at it.

    We aim to improve where we can given the spare time we have available.

    • Blair Conrad says:

      Hi, Shaun.

      Don’t worry I will. I just wanted to create a smaller sample that replicates the behaviour so you could have something manageable to work with. Currently I’ve only seen it while exercising the tests with a rather large assembly. Giving you that would probably make the Legal People sad and would be kind of mean…

      If you’re keen to just have a stack trace and brief description, I can still submit, though.

      Blair

  2. motrocker says:

    Installed a RC version today and ran a few of our tests and seen one of these AccessViolationException issues. Did you get to the root cause of this? It is a perfect tool I have to say, but only using a day now.

  3. Blair Conrad says:

    Ah, no. I could never get a small enough reproduction set to do anything useful with. Since it was just a couple of tests, I just left them uncovered.…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: