How To Perform Parallel Test Execution In TestNG With Selenium

The evolution in the process of software delivery in organizations in response to business agility has resulted in a paradigm shift from traditional release cycles to continuous release models. To achieve the product delivery objectives in this new paradigm, continuous testing plays a vital role in ensuring the quality of end-to-end processes, along with ensuring effective collaboration between Quality Assurance (QA) and development teams.

The new era of continuous testing demands QA teams to explore solutions to increase test execution speed. Enterprises are looking to leverage the benefits offered by automation testing, while ensuring that manual testing is used in scenarios where automation is not possible.

The traditional approach of manual testing is already taken over by automation testing. One popular automation testing strategy widely adopted by QA teams to further reduce the testing time while ensuring maximum test coverage is the shift from sequential testing to parallel testing. You can perform parallel test execution in TestNG or any other framework of your choice with Selenium.

Source

In this Selenium TestNG tutorial, we deep dive into the integral aspects of parallel testing in TestNG.

Let’s get started!

Introduction to Parallel Test Execution in TestNG

Parallel testing is an approach in which independent tests are run parallel to reduce overall test execution time. A tester runs two or more versions of the software concurrently with the same input or test methods. Each test can be run on multiple devices, browsers, and OS combinations.

The parallel testing process can be applied to a single application, multiple applications, or even multiple components of an application. Parallel execution in TestNG can be achieved at different levels – threads, methods, etc.

Parallel testing was introduced to replace the traditional approach of sequential testing. As the name suggests, in sequential test execution tests are executed one after another, which is more time-consuming.

Because of the growing demand for cross browser testing in today’s market, parallel testing is implemented heavily with Selenium automation testing.

Also Read – What is Selenium?

Advantages of Parallel Test Execution in TestNG

Parallel testing is a powerful approach that allows you to reduce the time taken to perform automation testing. There are various reasons that make it important for testers to adopt parallel testing in Selenium. Some of the most popular reasons are:

• Accelerated Test Execution Speed

If a sequential test takes 1 minute to complete a single test execution, it will take 5 minutes to run five tests one after another on the same device. However, overall test execution will take 1 minute (or less) when the same test is run in parallel on 5 devices (or input combinations).

• Increased Test Coverage and Wider Compatibility

Parallel Testing In TestNG enables you to test across more devices (desktop or mobile), browsers, and OS combinations at once. This extended test coverage will help in risk mitigation, reducing the chance of unnoticed bugs and offering wider compatibility, which will be instrumental in shipping a top-quality product.

• Overcome the Testing Bottleneck

Testing is often the bottleneck of DevOps. Tedious testing cycles can hold up the entire release process. But this bottleneck can be removed by accelerated test execution through parallel testing.

Also Read – Continuous Testing With DevOps

• Fix Bugs Earlier

Earlier the testing is completed, earlier the bugs can be unearthed and fixed. In addition, parallel testing is results-driven, since we can test more scenarios in less time, and there is more actionable data towards enhancements.

• Flexibility

You can revert to sequential testing when necessary.

Watch this video to learn how TestNG has become one of the most robust test automation frameworks and all you need to know to get started with TestNG in Selenium.

Challenges of Parallel Test Execution in TestNG

To achieve full-fledged parallel testing in TestNG, as with any type of testing, you may face some challenges. The common challenges that you will run into are often related to:

• Scalability
• Infrastructure limitations. This includes:
  • Availability of secure, enterprise-grade lab that meet all the requirements
  • Recurring investments involved in the maintenance and upgradation of test infrastructure.

The popular strategy recommended to overcome these challenges and move towards parallel testing is to use cloud-based testing providers. Through Cloud testing, teams can test their apps on devices and browsers hosted on the cloud infrastructure. By utilizing a cloud-based testing infrastructure, you no longer need to maintain in-house infrastructure.

Also, the scalability issues associated with the local testing infrastructure for covering all major browsers, their different versions, and OS will be taken care of. Needless to say, it is more cost-efficient as well.

Cloud-based cross browser testing platforms like LambdaTest offer a cloud Selenium Grid with access to 3000+ real browsers and devices for your testing needs. You can choose a specific OS-device-browser combination as per your requirement. The platform also aids you in performing parallel testing in TestNG, with Selenium, which can drastically scale your automation test and reduce test execution time through easy setup and minimal configurations. Here’s how you can perform cross browser testing using TestNG.

Also Read – Automated Parallel Testing In Selenium With TestNG

How to perform Parallel Test Execution in TestNG?

TestNG (where NG stands for ‘Next Generation’) is an open-source Java-based test automation framework that has gained huge popularity among developers and testers. The framework itself is inspired by JUnit, another popular unit testing framework in the Java ecosystem. However, it offers a variety of advanced and distinctive features, making it much more powerful than its peers. One such remarkable feature is the seamless support for Parallel Testing.

Here is a brief video to help you with Parallel Testing in TestNG.

The TestNG test automation framework allows you to run tests in parallel or multithreaded mode by utilizing the Multi-Threading concept of Java. Multi-Threading is the process of executing multiple threads simultaneously, without any dependence on each other. Therefore, exceptions occurring on one thread won’t affect the others. We enable parallel testing in TestNG by making required changes in the configuration file – TestNG XML file.

With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.

Here’s a short glimpse of the TestNG certification from LambdaTest:

However, you can follow the LambdaTest YouTube Channel and stay updated with the latest certifications and tutorials around Selenium testing, Cypress testing, and more.

Parallel Test Execution in Selenium using Configuring TestNG XML in TestNG

TestNG XML is the test suite configuration file in TestNG, which helps you organize your tests by defining test suites, grouping test cases in TestNG, and running them under one XML file. IN a nutshell, it helps in customizing the execution of tests.

It also allows you to run a single test file over numerous combinations of parameters and values specified in it. First, let us go through the key features of the TestNG XML file, which will be useful for this tutorial.

• Provides seamless parallel test execution
• Supports parameterization in TestNG using @Parameter annotation
• Allows grouping of test methods and group execution
• Allows executing test cases inside multiple Java class files
• TestNG listeners automatically create logs

A simple TestNG XML file without any configuration for the parallel tests will look like this.

To trigger parallel test execution in TestNG, i.e., run tests on separate threads, we need to set the parallel attribute. This attribute accepts four values:

• methods – runs all methods with @Test annotation in parallel mode
• tests – runs all test cases present inside <test> tag in the XML in parallel mode
• classes – runs all test cases present inside classes in the XML in parallel mode
• instances – runs all test cases in the same instance in parallel mode

We can also set the number of threads we wish to allocate during the execution using the thread-count attribute. TestNG has a default value of thread = 5.

Here, we have defined parallel test execution at suite level, and the @Test methods will start running simultaneously. The thread count is set to be 2, which means that only two threads will be created for this execution, on which 2 test methods will start running parallely. If there are more than 2 methods, all the remaining ones need to wait until any of the threads get free.

Now that we know how to configure TestNG for running tests simultaneously, it’s time to do the hands-on! In the upcoming section of this Selenium TestNG tutorial, we will learn how to create a TestNG framework setup and write tests and execute them parallelly through the online Selenium Grid.

Also Read – Create TestNG XML File & Execute Parallel Testing

Parallel Test Execution in multiple browsers using TestNG for Selenium automation testing

In this section, we will be writing test cases in the TestNG framework and learning how to execute them parallelly at tests, classes, and methods levels by configuring the TestNG XML file. Our test cases will be executed on 5 different browsers – Chrome, Firefox, Safari, IE, and Opera by leveraging the online Selenium Grid provided by LambdaTest. Also, a custom configuration will be done for the browser as well as the OS version. Using LambdaTest, you can now test on Safari browser online on both desktop and mobile browsers for cross browser compatibility.

If you are already working with TestNG, feel free to skip the installation and project set-up part, which we will be covering next.

Installation and Setup

Before getting started with the project setup, make sure that your system is equipped with the following prerequisites:

1. Java

   TestNG requires JDK 8 or higher. If you do not have Java installed in your system, you can download Java from the Oracle official website. Don’t forget to set the environment variables as well.

2. Eclipse IDE

   In this tutorial, for the demo, we will be using Eclipse as IDE. If not installed already, you can download Eclipse from the official Eclipse website.

3. Selenium Webdriver and Client for Java

   You can download by visiting the Selenium WebDriver official website. Selenium WebDriver is an open-source, free Selenium testing tool for browser automation.

Once this list is checked, the next step is to install TestNG in Eclipse. First, go to Eclipse Marketplace and install TestNG. Once installed, you can locate it under the Installed section.

You have now completed the installation. Let us do the basic project setup now.

Creating a TestNG Project

To use TestNG in a Java project, you must know how to create a TestNG project. You can follow the below-mentioned steps to create a new Java project in Eclipse.

Step 1: Launch Eclipse and create a new Java project by navigating to File > New > Java Project.

Step 2: Name your project and click Next.

Step 3: Add libraries by following the below-mentioned steps:

1. Right-click on the project folder and select Properties.
2. Choose Java Build Path.
3. Go to Libraries, select Classpath, and click Add Library.

Step 4: Select TestNG and click Next.

Step 5: Now, you will see that TestNG is added to the project library.

Step 6: Click Finish, and we have successfully created a TestNG project.

Step 7: Add Selenium API JAR files to the TestNG project by following the below-mentioned steps:

1. Right-click on the project and select Properties.
2. Select Java Build Path.
3. Under the libraries section, click on Add External JARs, which gets enabled upon opting for Classpath.



4. Select the option and upload the JARs for Selenium Java language bindings by navigating to the path where you have downloaded and saved it.



5. Once done, select Apply and Close.

With this, our project set-up is done, and we are good to start coding. Notice that the libraries are also visible in the left panel.

Also Read – How To Use Assertions In TestNG With Selenium

Watch this video to learn how to set up and use TestNG with Selenium to automate your testing process. We will also introduce the TestNG Priority method, which allows you to write easy-to-read and maintainable tests.

How to run test cases in parallel with Selenium in TestNG on Cloud Grid

In this section, we will learn how to achieve parallel test execution in TestNG by running the test cases on separate threads. Follow the below mentioned steps:

Step 1: Launch Eclipse and create a new Java project for TestNG. I have named it ParallelTestNG. You can skip this step if you have followed along with me in the Project set-up part.

Step 2: Create a TestNG class file by following the below-mentioned steps:

1. Right-click on project src and find TestNG towards the bottom of the list.
2. Select Create TestNG Class.



3. Name the class, choose the required annotations and click Finish. Here, I have named the class as TestCasesParallelExecution.

Let us quickly go through the TestNG Annotations that have been used:

• @Test – Implies that the method is a test case. Therefore, all tests should be marked with this annotation.
• @BeforeMethod and @AfterMethod – These annotations run once before and after each @Test method in a class.

Step 3: Write our first test case using TestNG and Selenium. Finally, it is time to get our hands dirty! In this section, we will be writing our first test case using TestNG. For this demo, I will be writing an automation test for a simple login scenario. Following are the test steps:

1. Launch the website URL (https://opensource-demo.orangehrmlive.com/)
2. Enter username and password.
3. Click on the LOGIN button.

Like mentioned before, we will be running this test case parallelly on five different browsers through an online Selenium Grid provided by LambdaTest.

Let us write our test case inside the TestNG class we created in the previous step. Follow along with me.

1. Create a free account in LambdaTest and secure your unique username and access key.

2. Inside the ParallelTest class(the TestNG class), define all the required variables for connecting with the Remote Grid. In addition, you will need to provide your unique username, access key.

3. Define automation setup steps inside the method marked by @BeforeClass annotation. This method gets executed first. Notice that I have also changed the method name from beforeClass() to setUp().

Inside the setUp(), we have defined the following:

• DesiredCapabilities for custom configuring our browser-device-OS version, enabling reporting and logging features, and naming the build. I have generated this by utilizing the Desired Capabilities Generator offered by LambdaTest.
• A New instance of RemoteWebDriver()
• Also, I have added @Parameters annotation on top of the method for receiving the browser, version, and platform values from our TestNG XML file.

4. Define tear-down steps inside the method marked by @AfterClass annotation. This method gets executed last. Notice that I have also changed the method name from afterClass() to tearDown().

5. Write our actual test steps. Our test methods will be marked by the @Test annotation. Since we are performing a login scenario, I have renamed the method to login().

I have made use of Chrome Developer tools for locating the elements.

Our final code looks like this:

Step 4: Configure the TestNG XML file. In this step, we will create the testng.xml file for our project and configure it to enable our tests to run in parallel mode. The steps are as below:

1. Right-click on our Project. Select New > File and name it as testng.xml.

2. Add the following boilerplate code to get started. You can name the test suite and test name as per your liking. But the class name should be exact as that of our TestNG class.

3. To trigger parallel test execution in TestNG, provide the parallel attribute to <suite>. As mentioned before, we can assign any of the four values given below, as per our requirement. For parallel execution of test methods, set the attribute value as “tests.”

4. Add parameters for configuring the tests to run in a specific browser, browser version, and OS. For example, below is the code for setting up the test case to run in Chrome Browser version 79.0 in Windows 10. Likewise, you can provide as many combinations as you like.

The final code for testng.xml looks like this.

We have a total of 5 different tags, and each of these will run on different threads.

5. Finally, it is time to run our test case. Right-click on our testng.xml file and navigate to Run As > TestNG Test Suite.

Our test starts getting executed. Go to your LambdaTest Automation Dashboard to see the live running status.

The test is executed parallelly in the 5 browsers, and reports and logs are generated successfully.

Watch this video to learn about the TestNG Annotations and how they help provide better structure and readability to the code.

How to run test methods in parallel with TestNG

Now let us see how to run test methods parallely in the TestNG. In this example, we will create one more method and execute them on different threads. The steps are as below:

Step 1: Create a new TestNG class file. You can also opt for an auto-defined TestNG XML file.

Step 2: Inside this class, I have added a new test method forgotPassword() which will:

1. Launch the website URL (https://opensource-demo.orangehrmlive.com/).
2. Click on Forgot your password? link.
3. Enter Username.
4. Click the Reset Password button.

So the final code will look like this:

Step 3: Configure the TestNG XML file. Set the value of the parallel attribute as ”methods”. My testng2.xml file now looks like this.

Step 4: Run the tests. Our two test methods start running in parallel. You can observe the live running status in your LambdaTest dashboard.

How to run test classes in parallel with TestNG

Lastly, let us see how to run test classes parallelly in the TestNG. First, let us create 2 classes that will be running parallel on separate threads.

1. Create 2 classes – TestClassOne and TestClassTwo. Each contains a test method.

TestClassOne.java

TestClassTwo.java

2. Create a new TestNG XML file and set the parallel attribute as “classes.”

3. Run your test suite. The test gets executed successfully.

Now if your test fail you can learn how to capture screenshot of failed tests easily through this video.

Conclusion

Parallel testing is widely adopted by Agile teams to radically reduce the time and cost associated with the traditional testing methods. It enables you to speed up the overall test cycle and deliver projects at a faster rate. To maximize the test coverage while ensuring scalability, the responsibility of maintaining a testing grid is often outsourced to cloud-based platforms.

In this Selenium TestNG tutorial, we learned how to achieve parallel test execution in the TestNG framework with Selenium. We have also leveraged the cloud-based Selenium Grid provided by LambdaTest to run our test cases parallely in 5 different browsers while performing Selenium automation testing with TestNG. I Hope this tutorial turned out to be beneficial to you. and If you are preparing for an interview you can learn more through TestNG interview questions. So, keep exploring different angles of parallel testing.

Happy Coding!