I’ve always wondered how to get code coverage information for a .net core app in my build pipeline in Azure Pipelines. When I just use the standard .net core template from AzD, I get my test results published but I don’t get any code coverage info. I’ve always known that getting code coverage is possible. I mean, you can customize Azure Pipelines to do ANYTHING, right? I say that almost every time I’m on stage so it’s gotta be true! Heh heh heh. I knew it could be done but I’ve been too lazy to figure it out. Well, enough people have asked me about this so today is the day I figure out how to do this.
Before we begin, here is my only criteria. I want a solution that will work anywhere. Not just on a windows machine. Not just with Visual Studio. This needs to work on anything where .net core works. After doing a quick search on the web, it looks like I can easily use Coverlet. A cross platform code coverage library for .net core with support for line, branch and method coverage. It also can output code coverage results in a variety of formats, including cobertura. Looking at the Azure Pipelines Publish code Coverage Results task, it looks like that task can publish Cobertura or JaCoCo results. Cool. We’ll use cobertura.
Setting Up Your Project
Ok, with that out of the way, the first thing we need is a unit test project. For this example, I created a MSTest project name CodeCoverageDemo.MSTest.Tests. Here are instructions on how to add a MSTest unit test project.
Next, add the coverlet.msbuild nuget package to your test project.
Your .csproj should now look like this:
Also, make sure the MSTest.TestAdapter is included in your testing project. I can’t remember if it automatically was added or if I manually added the nuget package.
But… that’s it. In terms of configuring your projects, all you need to do is add the coverlet.msbuild nuget package to your test project.
Testing out coverlet
If you want to see coverlet in action, drop into the command line, browse to your solution folder and run
dotnet test .\CodeCoverageDemo.MSTest.Tests\CodeCoverageDemo.MSTest.Tests.csproj
/p:CollectCoverage=true /p:CoverletOutputFormat=cobertura
Notice that not only does it figure out your code coverage information, it also creates the output file for you. Hooray for that!!! That’s what we need!
Setting up your build pipeline
Setting up your build pipeline is now pretty straight forward. First, we need to configure our Test task in our build pipeline
In the Arguments section, add the following (all one one line with no new lines)
–configuration $(BuildConfiguration) /p:CollectCoverage=true /p:CoverletOputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)\TestResults\Coverage\
Also, make sure the Publish test results and code coverage checkbox is checked. Next, add the Publish Code Coverage Results task under the Test
Next make sure the Code coverage tool is Cobertura, the Summary file is $(Build.SourcesDirectory)/TestResults/Coverage/*cobertura.xml (or wherever and whatever you called your summary file) and the Report directory is ($(Build.SourcesDirectory)/TestResults/Coverage (or again, wherever you outputted the code coverage results).
Save everything, queue up a build and….
Bam!!! Code coverage is now reported.
Supercharge you’re build report with a detailed code coverage reports page
Yay we got code coverage on our build reports. But… can we make it even cooler? Can we add a detailed code coverage reports page? After some more googling, guess what? You absolutely can.
The easiest way that I found is to use the ReportGenerator by Daniel Palm. First, add the ReportGenerator nuget package to your test project
Next, we need to modify the build pipeline by adding the ReportGenerator task from the marketplace to our pipeline after the Test task and before the Publish Code Coverage task.
Configure the ReportGenerator task by putting in the path to the Reports, the path to the Target directory, the Report type and the Source directory. For my example, since I “published” the code coverage report to $(Build.SourcesDirectory)/TestResults/Coverage, I set
$(Build.SourcesDirectory)/TestResults/Coverage/coverage.cobertura.xml for the Reports
$(Build.SourcesDirectory)/TestResults/Coverage for the Target directory
HtmlInline_AzurePipelines;Cobertura for the Report types
$(Build.SourcesDirectory) for the Source directories
Save it and queue up a build and our build summary looks like this
Notice we have a new Code Coverage tab. Either click on it, or click on the Code coverage succeeded link in the Summary and….
BAM!!!! How sick is that?
Conclusion
Adding code coverage to your .net core build is super simple. You need to add coverlet nuget package to your test project and you need to add ReportGenerator nuget package to your test project. And then you need to add 2 tasks to your build pipeline.
My sample project is here: https://github.com/abelsquidhead/CodeCoverageDemo
If you are a yml builds type of person, the yml for the build is in that repo here: https://github.com/abelsquidhead/CodeCoverageDemo/blob/master/azure-pipelines.yml
For more info on how to set up yaml based builds, see here.
Check it out, let me know what y’all think.
Thanks for this well illustrated post! I’ve written something very similar a few weeks ago except I use a YAML build definition and I install ReportGenerator as a dotnet global tool. I think it could be interesting for your readers:
https://www.meziantou.net/2019/02/18/computing-code-coverage-for-a-net-core-project-with-azure-devops-and-coverlet
Very cool! Thanks!
Awesome!
I got this working, but in using xunit I had to add another parameter to get it to run:
/p:Exclude=”[xunit*]*”
https://github.com/tonerdo/coverlet/issues/360#issuecomment-470044676
Thanks!
That’s awesome Carlo! Thanks!!
I hate that we have to add extra packages to get this working with Azure DevOps – TeamCity doesn’t make me do this with ReSharpers dotCover .. it just works! 🙁
Nevermind – i’ve realised you don’t have to do this after all! You can just add “–collect “Code coverage” as an extra command line argument in your .NET Core Test task and it will be published – woohoo!
This article is a little bit outdated now with .NET COre 3.1.
VS Test engine in azure devops now integrates into coverlet directly using the –collect “Xplat Code Coverage” command.
https://github.com/tonerdo/coverlet/blob/master/Documentation/VSTestIntegration.md