How To Generate Code Coverage Report Using JaCoCo-Maven Plugin

Code coverage is a software quality metric commonly used during the development process that lets you determine the degree of code that has been tested (or executed). To achieve optimal code coverage, it is essential that the test implementation (or test suites) tests a majority percent of the implemented code.

There are a number of code coverage tools for languages like Java, C#, JavaScript, etc. Using the best-suited code coverage tool is important to understand the percentage of code tested and take appropriate actions to ensure that you achieve the ideal code coverage!

For optimal code testing, many companies use the JaCoCo-Maven plugin that helps generate detailed code coverage reports. JaCoCo-Maven plugin is a free code coverage library for Java projects. It is based on the study of existing integration libraries that were created by the EclEmma team. At a larger extent, code coverage does give a brief overview of the product quality as higher the coverage, lesser are the chances of untested code getting into the release cycle.

SourceIn this blog, we will learn more about the JaCoCo-Maven plugin and how this plugin is implemented using Maven. We will also integrate a Maven project and generate detailed code coverage report for the tests by performing Selenium.

What is Code Coverage?

Code Coverage is a metric used in software development to represent the extent to which an application’s source code is executed while a test suite is run. To see and evaluate the code coverage of a software application, a report is produced. Then, code quality may be ensured using this code coverage report.

The JaCoCo-Maven plugin is used to generate code coverage reports. Source code with high code coverage has more of its code executed during testing. For example, if the software you are testing contains 100 lines of code and the number of code lines validated in the software is 90, then the code coverage percentage of that software application will be 90%.

Benefits of Code Coverage

Code coverage is a very useful metric for developers, testers, and QA engineers. Here are some of the salient benefits of code coverage:

1. Code coverage reports provide useful insights in detection and elimination of dead code. This can be avoided by following implementation best-practices which in turn results in improved code maintainability and better product quality.
2. Quality assurance can help detect code that has not been covered using the test implementation.
3. Developers can finish the software development process faster, increasing their productivity, scalability, and efficiency. This results in reduced Time to Market (TTM).

As we know, code coverage is very important for every software product. However, if you are a developer or a tester, you can cure your interest in code coverage via the article: Code Coverage vs Test Coverage, which one is better?

Now that we have done a quick recap about the integral aspects of code coverage, let’s deep dive into our core topic i.e. generating code coverage reports using Jacoco Maven plugin.

What is JaCoCo-Maven Plugin?

JaCoCo-Maven (abbreviation for Java Code Coverage) plugin is an open-source code coverage tool for Java. It creates code coverage reports and integrates well with IDEs(Integrated development environments) like Eclipse IDE.

It also integrates smoothly with CI/CD tools (e.g. Jenkins, Circle CI, etc.) and project management tools (e.g. SonarQube, etc.). It is a part of the Eclipse Foundation and has replaced the EclEmma code coverage tool in Eclipse.

How does JaCoCo-Maven Plugin work?

1. 1. The JaCoCo-Maven plugin runs the coverage by instrumenting Java code through a runtime agent. In simple terms, you attach this agent to a JVM (Java Virtual Machine) when it starts. This agent is termed as JaCoCo agent. The first execution start-agent starts this JaCoCo Runtime Agent.
   2. Whenever a class is loaded, JaCoCo can instrument the class so that it can view when the class is called and what lines (of code) are called during the testing process. By keeping this track, it builds up the code coverage statistics which is done on the fly during the second execution (i.e. generate-report).
   3. By default, the file is created as soon as the JVM terminates but it is also possible to run the agent in server mode. This triggers a dump of the results and the report is created before the termination. Shown below are the internals of the JaCoCo plugin:

1. You can define goals and rules in the configuration of the JaCoCo-Maven plugin. This offers you the flexibility to set limits and helps in checking the amount of code coverage.
2. Maven-surefire plugin is the default Maven plugin. This runs the tests in JVM and provides coverage reports. While the JaCoCo plugin instruments the code already executed by a plugin (e.g. Surefire plugin). Thus, it is a good practice to check for the dependency of the maven-surefire plugin.

 

Read – Getting Started With Maven for Selenium

 

Why is Jacoco Maven Plugin good for code coverage?

The JaCoCo-Maven plugin is appropriate for code coverage because of the following reasons:

1. While working on any project, developers mostly prefer IDEs because it simplifies the coding and testing experience. JaCoCo can be installed on Eclipse IDE in the name of EclEmma, by downloading EclEmma from its marketplace.
2. It is easy to add the JaCoCo plugin to all types of builds, including ANT, Maven, and Gradle. It can also be integrated with CI/CD tools like Jenkins, Circle CI, etc. This makes it versatile for a lot of use cases.
3. The code coverage report generated by JaCoCo is a simple and informative HTML file that can be viewed in any browser or IDE.
4. JaCoCo also provides offline instrumentation (i.e., all the classes are instrumented before running any tests).
5. Analysis of a report is also quite easy, as it is color-based and provides the exact percentage of code coverage.

How to set up JaCoCo Plugin with Maven?

To get code coverage reports in a Maven project, we first need to set up the JaCoCo Maven plugin for that project. By integrating the JaCoCo plugin, the results of the code coverage analysis can be reviewed as an HTML report. The current version of the JaCoCo-Maven plugin can be downloaded from the MVN Repository.

Here are the steps to integrate JaCoCo Maven plugin with a Maven project:

1. Every Maven project has a pom.xml file, used to declare all the dependencies and plugins. The JaCoCo-Maven plugin is declared in the same POM.xml file. The XML code for the same is :

This is the basic XML code added under the build tag for specifying the JaCoCo plugin in a Maven-based project. We can enhance the functionality (like mentioning when a report should be generated, etc.) by specifying goals and rules in the execution tag.

2. After the version tag, we add the execution tag. This tag prepares the properties or execution to point to the JaCoCo agent and is passed as a VM (in this case, JVM) argument.

3. For running simple unit tests, two goals set in execution tags will work fine. The bare minimum is to set up a prepare-agent and report goals.

• Prepare-agent goal: The prepare-agent goal prepares the JaCoCo runtime agent to record the execution data. It records the number of lines executed, backtraced, etc. By default, the execution data is written to the file target/jacoco-ut.exec.
• Report goal: The report goal creates code coverage reports from the execution data recorded by the JaCoCo runtime agent. Since we have specified the phase property, the reports will be created after the compilation of the test phase. By default, the execution data is read from the file target/jacoco-ut.exec, and the code coverage report is written to the directory target/site/jacoco/index.html.

4. For running simple unit tests, the above configuration works fine. However, we would need constraints to be put on the code coverage reports (e.g. specify the destination directory, etc). This can be done via a configuration tag.

Configuration of first execution :

As per the above code, you can see some tags like destFile, etc., are specified. Here is a brief description of the tags:

• destFile tag: The destFile tag is used for setting the path to the file containing the execution data.
• propertyName-surefireArgLine tag: This tag sets the name of the property that contains the settings for the JaCoCo runtime agent. This also sets the VM argument line when the unit tests are run.

Configuration of second execution :

As per the above code, you can see tags like dataFile, etc., are specified. Here is a brief description of the tags:

• dataFile tag: The dataFile tag is used to set the path to the file containing execution data.
• outputDirectory tag: This tag sets the output directory for the code coverage reports.

5. We can also add rules to our configuration tag to keep a check on the code coverage percentage. This can be done as shown below:

Configuration of third execution :

Here, a new goal check is defined. The jacoco:check goal is bound to verify the rule specified. Rule is given in the rule tag. You have the flexibility to specify more than one rule.

• Element tag: This tag specifies the element on which the rule has to be applied.
• Limit tag: Tags like counter, value, minimum, etc., are used to limit the code coverage percentage. A command like mvn clean verify can be used for ensuring whether the rule is followed or not.

6. There are multiple goals and rules which can be defined in the JaCoCo-Maven plugin configuration.

Code Coverage Reports Using Maven and JaCoCo Plugin

In this section, we would demonstrate the steps to generate code coverage report using the JaCoCo Maven plugin. The demonstration is done by taking a very simple test scenario. So, let’s get started.

Also, watch this in-depth tutorial to learn how to measure code coverage in Java using the JaCoCo plugin.

Pre-requisites:

1. Maven 3.0 or higher: Maven is a project development management and comprehension tool. You can install the latest version from the official Apache Maven website.
2. Java 1.5 or higher: You need to ensure that Java is installed on your machine. In case you do not have Java installed, please download the latest version of Java from the official Java website.
3. Eclipse IDE for Java Developers: Though you can use the IDE of your choice, for this demo we have used the Eclipse IDE.

Steps to create a simple Maven Project:

1. 1. In Eclipse IDE, Go to File>New>Maven Project.

1. 1. A new dialog box appears. Make sure that the ‘Use default Workspace location’ checkbox is ticked. Click Next.

1. 1. For selecting Archetype in the project, type org.apache.maven in the textbox that is located next to the Filter. Select maven-archetype-quickstart and click Next.

1. 1. Now, specify the Group Id as com.example and Artifact Id as jacoco-example. The artifact Id is our project name. Finally, click on the Finish button.

1. 1. You can see the project file and folder hierarchy in Project Explorer.

How to specify JaCoCo Maven plugin in POM.xml

1. 1. Open POM.xml, scroll to tag. We will specify the JaCoCo-Maven plugin in the section that lists the Maven plugins. Just copy the below code and paste it above the tag.

1. 1. Since we are demonstrating report generation for automated web testing with JUnit, we will also declare the JUnit dependency in POM.xml.Here is the entire content of POM.xml:

1. From the project directory, traverse to com.example.jacoco_lambdatest package existing in src/main/java. Create a new Java class named LambdaTest.java. We will write a simple setUp() function in it that provides the desired capabilities of Selenium Grid.

Adding JUnit test Cases in the project:

1. 1. We will create a simple JUnit test case in AppTest.java. This is provided by default, in the src/test/java under the package name com.example.jacoco_lambdatest.

Generating code coverage reports:

1. 1. Just click on the Run As button and set the configuration as Maven Test.

1. 1. Instead, you can open cmd(Command Line), traverse to the project folder, and run the maven command, “mvn test.”

1. 1. Running the JUnit tests will automatically set the JaCoCo agent in motion. It will create a report in binary format in the target directory, with path target/jacoco.exec. The output of jacoco.exec cannot be interpreted single-handedly but other tools like SonarQube and plugins can interpret it. As we have earlier specified the jacoco:report goal, it will generate readable code coverage reports in popular formats like HTML, CSV, and XML.
   2. As the build is successful, go to the target folder, then to the site>jacoco folder. The code coverage report (i.e. index.html) is located in target/site/jacoco/index.html. The report looks like this:

1. 1. You can drill down at a micro level by clicking on com.example.jacoco_lambdatest>LambdaTest in the report.

1. 1. By clicking on specific functions, you will have a more detailed view in LambdaTest.java.

1. Here, you would see a lot of diamonds of different colors like green, yellow and red. These are the specifications used in the report to symbolize which line of code was executed and when it was executed. We will learn more about it in the next section of the report analysis. With this, you have successfully generated a code coverage report via the Jacoco Maven plugin.

Analysis of the Code Coverage Report

Our code coverage report shows 94% instruction coverage, 100% branch coverage, which is a great code coverage score. Later, we will try to achieve a 100% code coverage score by adding more test cases.

The 38 instructions shown by JaCoCo in the report refer to the bytecode instructions instead of Java code instructions. The JaCoCo reports help you visually analyze
code coverage by using diamonds with colors for branches and background highlight colors for lines. A brief explanation of the diamonds seen in the code coverage report is below:

• Red diamond indicates that no branches have been exercised during the testing phase
• Yellow diamond indicates that the code is partially covered (i.e. some branches are not exercised)
• Green diamond indicates that all branches are exercised during the test

The same color code applies to the background highlight color for the line coverage.
The report mainly provides three crucial metrics :

1. Line coverage: This reflects the amount of code exercised based on the number of Java byte code instructions called by the tests.
2. Branch coverage: This shows the percentage of exercised branches in the source code. These are typical if/else or switch statements.
3. Cyclomatic complexity: This reflects code complexity via the number of paths needed to cover all possible paths in a code. It also refers to the number of test cases needed to implement to cover the entire code. As there is no switch or statement in the code, the cyclomatic complexity will be 1; only one execution path is sufficient to cover the entire code.

Introducing more test cases for improving code coverage

1. 1. To achieve better code coverage, more tests need to be introduced that test the code that was not covered earlier via the test implementation.
   2. Go to AppTest.java in src/test/java to add more test cases.
   3. The new test cases added to AppTest.java will look as follows:

1. 1. Let’s run the maven jacoco:report to publish a new coverage report.

1. 1. JaCoCo offers a simple and easy way to track code coverage score by declaring minimum requirements. Build fails if these requirements are not met else the build is successful.
   2. These requirements can be specified as rules in POM.xml. Just specify the new execution specifying ‘check’ goal in POM.xml. Add the below code after the second <execution> tag in POM.xml.

1. 1. With this, we are limiting our coverage ratio to 50%. This signifies a minimum of 50% of the code should be covered during the test phase.
   2. You can run maven clean verify to check whether the rules set in jacoco:check goal are met or not.

1. The log shows “All coverage checks have been met.” as our code coverage score is 94% which is greater than our minimum 50%.

Automation Testing On LambdaTest Selenium Grid using Maven Project with Jacoco Plugin

Selenium testing on the cloud helps you attain better browser coverage, increased test coverage, and accelerated time to market. Parallel testing in Selenium helps you achieve the above mentioned requirements.

LambdaTest Cloud Selenium Grid is a cloud-based scalable Selenium testing platform that enables you to run your automation scripts on 2000+ different browsers and operating systems.

Pre-requisites:

To run the test script using JUnit with Selenium, first, we need to set up an environment. You would first need to create an account on LambdaTest. Do make a note of the username and access-key that is available in LambdaTest profile section.

We will use this sample project for Java Selenium testing.

Importing project to Eclipse IDE:

After downloading a zip file of the project: junit-selenium-sample from GitHub, we import it to Eclipse IDE by following the below mentioned steps:

1. 1. Go to your Eclipse IDE, click on the File menu and select Import. A new dialog box appears.
   2. Type Maven in the textbox below and select Existing Maven Projects, and then click Next.

1. 1. In the next dialog box, click on Browse and traverse to the project folder. Also, tick the checkbox giving the path to the POM.xml file, and click Finish.

1. Your project will be loaded in Eclipse IDE successfully.

Adding dependencies in the POM.xml file:

1. 1. Open the POM.xml, now add the dependencies of JUnit, Selenium, and JaCoCo Maven Plugin. After adding the dependencies to the code of POM.xml should look like this:

Configuring the Desired Capabilities for JUnit Automation Testing:

1. 1. To connect to LambdaTest Selenium Automation Grid, the first thing done is invoking a remote webdriver. This remote driver requires some capabilities like browser, browser versions, operating system, etc., to build an environment. The code to it looks as follows:

1. 1. In the JUnit automation testing code, the capabilities like browser, browser versions, operating system information, etc., can be customized and are passed via Capabilities Object.
   2. LambdaTest has made this process very easy by providing an inbuilt Capabilities Generator. The Capabilities Generator will automatically generate the code for desired capabilities based on your input. Such as, our configurations are:

Fields	Selected Values
Operating Systems	Windows 10
Browser	Chrome
Browser Version	62.0
Resolution	1024×768
Selenium Version	3.13.0

1. Selecting the above-specified configuration in the Capabilities generator and paste it to LambdaTestBseTest.java.

Specifying LambdaTest Username and Access Key in required Java Class:

1. 1. In the Project Explorer, you would see 3 Java classes: LambdaTestBaseTest.java (contains the setup required for Java Selenium testing), Parallelized.java (contains Junit tests for parallel testing on LambdaTest Selenium grid), and SimpleTest.java (contains simple unit tests).

1. LambdaTestBaseTest.java fetches the required data like desired capabilities, username, and access key from a config file. This is provided in src/test/resources as config.json.
2. Specify the desired capabilities, Username, and Access key in config.json. This JSON file is used as you can provide multiple configurations in it for realizing parallel testing with Selenium. to specify multiple configurations in config.json and then fetch them later.

Unit Testing using JUnit with Selenium:

1. 1. SimpleTest.java is the Java class for specifying a single unit test case for testing and performing code coverage using the JaCoCo Maven plugin.

1. 1. This is a simple Selenium WebDriver test that will open a sample to-do application that will do the following task:
      • Mark the first two items as mark done.
      • Add a new item to the list.
      • Return the added item.
   2. Trigger the command mvn test on the terminal to build and run the test case.

1. 1. Now, login to your LambdaTest Account and go to Automation. You will find the tests you ran under the build name “JUnit JaCoCo Tests.”

1. 1. Click on “JUnit JaCoCo Tests” and review them in detail. LambdaTest has recorded the video. So you can see the visuals too.

Generating Code Coverage Report via JaCoCo Maven Plugin:

1. Now, as we have run the JUnit test cases on LambdaTest Selenium Grid, the code coverage report should be generated via the JaCoCo Maven plugin.
2. Just go to the target folder, you will find the binary format of the report as jacoco.exec.
3. You can view the HTML, CSV, and XML form of the code coverage report in the target/site/jacoco folder as index.html, jacoco.csv, and jacoco.xml, respectively.
4. Great! Now you can also try to improve the code coverage score and analyze the code coverage report generated.

Conclusion

In this article, we have seen how to use the JaCoCo-Maven plugin to generate code coverage reports for Java projects. We have also leveraged the agility and scalability of LambdaTest Selenium Grid cloud to automate the test processes. Remember, though, 100% code coverage is not responsible for reflecting effective testing, as it only shows the amount of code exercised during tests.

Yet, it helps to reduce the number of bugs and improves the software release quality. Also, it adds minimal overhead to the build process and allows it to keep a certain threshold as the development team adds edge cases or implements defensive programming.

SourceCode coverage is a really good practice, especially when you have conditional statements in your source code. Hopefully, this article will help you learn how to achieve code coverage with JaCoCo for a Maven project. That’s all for the day. However, we have some very useful content coming up on the LambdaTest blog, so stay tuned and happy testing!