Complete Tutorial On Appium Parallel Testing [With Examples]

In today’s fast-paced world, the primary goal of every business is to release their application or websites to the end users as early as possible. As a result, businesses constantly search for ways to test, measure, and improve their products. With the increase in competition, faster time to market (TTM) has become vital for any business to survive in today’s market. However, one of the possible challenges many business teams face is the release cycle time, which usually gets extended for several reasons.

Different techniques and approaches are tried to check the business requirement, development, quality, and overall progress to see all possible things required to overcome the challenges. All of this ensures expediting the time to market without compromising a wee bit on the quality front.

Following are some of the reasons which I have faced in my experience while working with many projects:

• Coding issues.
• Automation pipeline takes longer than expected to run, which doesn’t help with faster feedback.
• Build generation takes a lot of time, which causes a delay in running automated regression testing and performing manual exploratory testing at the end.
• Flakiness of the tests.

And likewise, there might be much more to list. However, I have listed a few common in many teams.

From the reasons listed above, let’s talk about why the Automation pipeline takes longer than expected to run, which doesn’t help with faster feedback, and also take a look at the possible solution of how that can be minimized.

Consider a scenario of a mobile e-commerce application built for Android and iOS platforms, and you need to run the automation regression test suites on both platforms. Each test suite has around 100+ test cases to run. So considering we have two platforms, iOS and Android, we need to run 200 tests, after which only we would be able to provide feedback for the builds.

Consider a single test taking around 10 seconds to run, so considering 100 test cases, it would take 1000 seconds to run a single suite while performing Android automation testing and another 1000 seconds to perform iOS automation testing.

This is where parallel testing comes to the rescue! In further sections of this Appium testing tutorial, I will discuss parallel testing and show how to perform Appium parallel testing.

So get ready to take this journey of running automation tests with Appium parallel testing with me. If you are preparing for an interview you can learn more through Appium Interview Questions.

What is Appium?

Before we deep dive into understanding more about Appium parallel testing, let’s first understand Appium as without using this framework, we won’t be able to proceed with writing the mobile automation test.

Appium is a popular open-source mobile app testing framework that helps automate the tests for iOS, Android, and Windows platforms. It can be used to test native, mobile web, and hybrid apps. You can learn more about these apps from this blog on web vs hybrid vs native apps.

It supports testing the react native apps out of the box and runs the test cases using the WebDriver interface. It also supports multiple programming languages like Java, Javascript, C#, Python, PHP, and Ruby.

Appium can be used to run tests on real devices on local as well as on cloud platforms and also run the tests in parallel using TestNG. If you want to learn more about Appium and TestNG, you can go through this Appium with TestNG tutorial.

Learn about the Appium testing trends that matter and will shape the future of the testing landscape.

Watch this video to learn about Appium, one of the most popular open-source test automation frameworks for mobile app testing.

What is Parallel Testing?

Parallel testing is the way to execute multiple automation tests simultaneously, where tests are run on different threads and consume different resources. We can run different tests on different platforms, multiple devices, and with different inputs simultaneously. So, in the example of the e-commerce application above, it took 1000 seconds to run all tests on Android and again 1000 seconds to run on iOS. So, in total, we would need 2000 seconds to run the tests on Android and iOS devices.

Let’s consider that there are multiple devices for Android and iOS; the time will be multiplied by the number of devices you add to run the tests on. By running the tests in parallel, we could have saved 1000 seconds as both would have run simultaneously, providing us with faster feedback.

Let’s discuss in detail how to perform Appium parallel testing using cloud-hosted mobile platforms like LambdaTest and how it can help you reduce the time involved in running the tests.

If you look closely at the above figure, it shows the orchestration of the tests of how they are designed to get executed on Device 1 and Device 2, respectively, when they are run serially.

Device 1 may be an Android device, and Device 2 might be an iOS Device or another Android device. It all depends on how you want to run your tests. Once the tests are executed, Device 1 will be set up, and Test 1 to Test 4 will be executed on it.

Once the execution finishes on Device 1, only Device 2 will be picked for executing another set of the same tests. Here, the time taken to run tests on Device 2 may depend more on the waits, locator strategy, etc., depending upon the device and platform used for running the tests.

Now, let’s talk about running the mobile tests in parallel. The above figure shows that all the tests are executed parallely on multiple devices. As soon as you execute the tests, all 4 devices are set up simultaneously. These 4 devices might be on the same platform like iOS or run on different platforms. The goal is to execute all the tests at once so we save time and get quick feedback on the tests.

Advantages of Appium Parallel Testing

By now, you have a fair idea about what Appium parallel testing means and why we use it. Now, let’s discuss some advantages of Appium parallel testing.

The following are some of the advantages of performing Appium parallel testing:

• Reduces the overall execution time of the tests.
• Provides leverage to run tests on multiple devices with different platforms and platform versions.
• Accelerated feedback on the builds.
• Fast feedback helps the team to fix the issue early and remove the bottlenecks.
• Achieve faster and more frequent releases while keeping pace with Continuous Integration and Development.

Let me give you one recent example from one of my previous projects where we worked on a built-in mobile application for Android and iOS, and we used Appium to run the mobile tests. The CI/CD pipeline was built using Test Pyramid, so the Unit Tests ran first, after which the Contract tests followed. After that, the Integration tests ran, and finally, the end-to-end regression tests were executed. The builds were deployed to the QA environment after all the tests were successfully run, after which the QAs performed manual exploratory testing on the builds.

Around 7 User journeys were involved, and mobile tests were run on Android and iOS platforms using Appium. It took around 20 minutes to run the tests on Android devices only. Once all the Android tests were run, the iOS tests were triggered, which again took around 20-25 mins to complete the execution of the iOS tests. So, ultimately around 40-45 minutes were required to perform the end-to-end testing.

In between, we faced some flakiness issues, and sometimes tests started failing. We had to bear the brunt as we couldn’t get the build on time to perform manual exploratory testing, which eventually led to a spillover of the stories.

The team came up with the idea of running the mobile tests through Appium parallel testing. Our real device criteria selection for mobile tests was as follows:

1. Latest and real Android and iOS devices with the latest platform version.
2. Android and iOS real devices with the respective older platform version (this was to check the minimum version support for the app – we supported Android 8 as the minimum version and iOS 12).

And we configured and updated our mobile tests to perform Appium parallel testing, which helped us reduce the test execution time by 50%! All the end-to-end tests were executed in 20-25 minutes, and we got faster feedback on our tests now.

Since end-to-end tests depend on each other, we need to design the tests so that they can run in parallel successfully. TestNG has the dependsOnMethods attribute for tests, which could be used for chaining the tests for the end-to-end journey. For e.g., Product Purchase tests should run only after login is successful, and Payment should happen only if the Product checkout is successful. Such tests should be carefully designed to run while performing Appium parallel testing.

“Harness the power of Appium for parallel testing with ease. For detailed Appium command references, check out our ‘appium-commands-cheat-sheet’ to streamline your parallel testing efforts. Learn how to optimize your testing strategy and overcome challenges with Appium in this comprehensive guide.”

Note: Boost your Testing efficiency by ultimate guide of Appium Commands Cheat Sheet for Developers.

How to perform Appium Parallel Testing on LambdaTest?

LambdaTest is a reliable, scalable, secure, and high- performing Appium testing platform that empowers development and testing teams to accelerate their release cycles. You can automate your mobile apps on a real device cloud of 3000+ real iOS and Android devices.

You can also subscribe to the LambdaTest YouTube Channel and stay updated with the latest tutorials around Selenium testing, Cypress E2E testing, CI/CD, and more.

The following are some of the features of LambdaTest’s Appium testing platform:

• Online device farm to run App test automation on real devices.
• Parallel testing to reduce test execution by 10X.
• Test your native application features, gestures and interaction.
• Geolocation testing and Localization testing of mobile applications.
• Enterprise-ready LambdaTest tunnel to test locally hosted applications.

Let me tell you about the Android and iOS apps we will use to write the tests. These are demo apps built by LambdaTest:

• Proverbial App on Android.
• Proverbial App on iOS.

In case you are a beginner to Appium you can learn to perform mobile app testing easily through this tutorial.

Before we begin writing the tests, let me tell you about the tools we would be using to write and run the tests:

The following programming language and tools have been used in writing and performing Appium parallel testing:

• Programming Language: Java 11
• Web Automation Framework: Appium (Appium Java client – 7.6.0)
• Test Runner: TestNG
• Build Tool: Maven

Cloud Platform for test execution: LambdaTest

Let’s discuss the mobile testing strategy for the Proverbial app and focus on what and how we automate it?

This app opens with a Home Page, a welcome message, and some buttons.

We would use the following test scenario for Appium automation:

1. Open the App and check the welcome message – “Hello! Welcome to lambdatest Sample App called Proverbial” is displayed correctly.
2. Click the Text button and check that the text message “Proverbial” is displayed.
3. Click the Notification button and check that the notification is displayed on the top. Check that the notification appears correctly and click to close it.
4. Click the Toast button and check that the toast message is displayed at the bottom of the screen, and verify its text “Toast should be visible.”
5. Click the GeoLocation button to check that the app navigates successfully to the geolocation page. Once navigation to the geolocation page is successful, navigate back to the home page.
6. Click the SpeedTest button to check that the app navigates to the Speed Test page, verifying the banner is displayed. After verification, navigate back to the Home Page.
7. Click the Browser menu at the bottom of the screen. Once the app navigates to the browser page, enter the text “https://www.lambdatest.com” and click the Find button to check if the website loads on the screen.

Appium Parallel Testing Strategy:

As mentioned in the initial part of this blog on Appium parallel testing, we would be running all the tests in parallel as follows:

Test Case No.	Platform Name	Platform Version	Device Name
1	Android	10	Galaxy S9 Plus
2	Android	11	Pixel 5
3	iOS	14	iPhone 12 Pro
4	iOS	14	iPhone 11

Getting Started with Appium Parallel Testing

As discussed earlier, this project has been created using Maven. TestNG is used as a test runner. Once the project is created, we need to add the dependency for Appium, TestNG, lombok, and jackson-databind in the pom.xml file.

Pom.xml

For reading JSON we would be using the JsonNode class from the jackson-databind dependency. It has some rich features and allows us to read the JSON file seamlessly with less code to write. An appropriate approach and logic must be applied to get the respective values.

Appium Java client dependency is added, which will help us in writing the mobile automation tests.

Versions of the dependencies are set in a separate properties block. This is done for maintainability, so if we need to update the versions, we can do it easily without searching the dependency throughout the pom.xml file.

properties block

You can learn more about performing app automation on real devices through this video:

How does TestNG help in performing Appium Parallel Testing?

TestNG is a Java-based test automation framework that helps run the tests. Here, “NG” stands for “Next Generation.” It is designed to make end-to-end testing easy for the testers. TestNG allows performing Appium parallel testing by setting the “parallel” attribute in the “” tag respectively to “tests”, “methods”, “classes”, and “instances”.

As per the setting to run the tests, it will start executing them in separate threads.

You can use the following options to run Appium parallel testing with TestNG:

parallel=”tests”: Providing this value for the parallel attribute in the tag will run all the methods in the same tag in the same thread but each tag will be in a separate thread.

parallel=”methods”: Providing this value for the parallel attribute in the tag will run all your test methods in separate threads. Dependent methods will also run in separate threads but will respect the order you specified.

parallel=”classes”: Providing this value for the parallel attribute in the tag will run all the methods in the same class in the same thread but each class will be run in a separate thread.

parallel=”instances”: Providing this value for the parallel attribute in the tag will run all the methods in the same instance in the same thread but two methods on two different instances will be running in different threads.

As we saw, there are multiple options provided by TestNG. However, it all depends upon your test design and orchestration to run your tests in parallel.

Automation Tests

I won’t be discussing finding the locators and writing the automated tests for iOS and Android in this blog on performing Appium parallel testing. Please refer to my earlier blog on testing React native apps. I have discussed finding locators for iOS and Android apps created using the react-native framework using Appium Inspector.

In-depth details about how to write tests using Appium and Page Object Model are also provided in the above blog. Moreover, I have also provided the code walkthrough of creating pages and tests using Appium with Java and how to run it using Maven.

Getting Started with Appium Parallel Testing with TestNG

Let’s discuss the strategy we defined for running the tests in parallel. To start with, a configuration file in JSON format has been created and placed in the test/resources folder from which we will be reading the Desired Capabilities required to run the tests on LambdaTest real device cloud.

Configuration (FileName – parallel.config.json)

All the common desired capabilities have been put in the “commonCapabilities” JSON Object. These capabilities are required by the LambdaTest platform for getting the tests run on real devices and also enabling the logs, respectively.

The other block you see for “deviceAndBuildCaps” is required to tell the test and its respective driver on which platform, device, and OS version of the platform and on which app to run the tests.

Uploading the App to LambdaTest

Value for the app is the app_url we get after uploading the app to LambdaTest using its app-upload API.

There are two ways to upload the API to LambdaTest: using App File and using App Url. The following steps must be taken for uploading the app on LambdaTest.

1. Using App File:

App can be uploaded using the following curl command:

• MacOS / Linux Users:
curl -u "YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESS_KEY" \ --location --request POST 'https://manual-api.lambdatest.com/app/upload/realDevice' \ --form 'name="Android_App"' \ --form 'appFile=@"/Users/macuser/Downloads/proverbial_android.apk"'

1 2 3 4

curl -u "YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESS_KEY" \ --location --request POST 'https://manual-api.lambdatest.com/app/upload/realDevice' \ --form 'name="Android_App"' \ --form 'appFile=@"/Users/macuser/Downloads/proverbial_android.apk"'

• Windows Users:
curl -u "YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESS_KEY" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk""

1	curl -u "YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESS_KEY" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk""

2. Using App URL:

App can be uploaded using the following curl command:

• MacOS / Linux Users:
curl -u "YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESS_KEY" --location --request POST "https://manual-api.lambdatest.com/app/upload/realDevice" --header "Content-Type: application/x-www-form-urlencoded" --data-urlencode "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" --data-urlencode "name=Proverbial_App"

1

curl -u "YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESS_KEY" --location --request POST "https://manual-api.lambdatest.com/app/upload/realDevice" --header "Content-Type: application/x-www-form-urlencoded" --data-urlencode "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" --data-urlencode "name=Proverbial_App"

• Windows Users:
curl --location --request POST "https://manual-api.lambdatest.com/app/upload/realDevice" -u "YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESS_KEY" --header "Content-Type: application/x-www-form-urlencoded" --data-urlencode "url=https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" --data-urlencode "name=test" --data-urlencode "custom_id=my_app"

1

curl --location --request POST "https://manual-api.lambdatest.com/app/upload/realDevice" -u "YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESS_KEY" --header "Content-Type: application/x-www-form-urlencoded" --data-urlencode "url=https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" --data-urlencode "name=test" --data-urlencode "custom_id=my_app"

Once the App is uploaded successfully, we should receive “app_url” in response, which could be used for running the tests.

Code Walkthrough:

Reading the JSON config file and setting the Desired Capabilities.

Configuration (FileName – DriverManager)

GitHub

Since we are reading the desired capabilities from the JSON file it is necessary to parse the JSON file and get all the details from it so we could set those values in the desired capabilities, which is required to run the tests. JsonNode class from jackson-databind dependency is used for reading the JSON file.

The JSON configuration file has two blocks: “commonCapabilities,” a JSON object, and “deviceAndBuildCaps,” a JSON Array that holds a different set of values per the devices we need to run the test on.

First, we will set the deviceAndBuildCaps. A variable deviceId is created, which will take the value as per the Parameter value set in the testng.xml file. Once we have the value, we will compare this value with the value we would provide in the id field of the parallel.config.json file.

Once this is done, we have the proper block of desired capabilities assigned to the respective tests we are planning to execute. So, suppose we plan to run the tests on an Android device. In that case, the correct block of desired capabilities should be picked from the configuration file and mapped to the desired capabilities to run the tests successfully.

After the deviceAndBuildCaps, we would be setting the commonCapabilities which are required as per the LambdaTest platform for enabling the logger and using real devices for testing.

Finally, these capabilities would be set in the AppiumDriver, which will eventually use it to run the tests respectively.

The most important task is to set up the drivers so we can use them to run the test. Here, AppiumDriver is used to run the tests as we would be running the tests on the
LambdaTest platform as AndroidDriver and IOSDriver both extend AppiumDriver.

ThreadLocal class is used for setting the driver for running the tests as it is threaded safe and works very well when tests are run in parallel. The main reason behind using ThreadLocal is that two threads cannot see each other’s ThreadLocal variables even if two threads set different values on the same ThreadLocal object. It will help us in running the parallel tests safely.

So, once desired capabilities are set, we need to set the driver and pass the LambdaTest Username, Access Token, and Grid URL along with the desired capabilities so it can be used to generate the connection with the cloud platform to run the tests.

We would set the Username and Access Token by taking it from the user as an environment variable. From a security point of view, the values of these attributes should not be disclosed. Hence we would be setting it on run time. Grid URL will be set using a static variable in the code itself, as this value will not change at any time in the execution of the code.

One important point to note here is that Lombok’s @Builder annotation is used in this class, allowing us to build and take the device Id value on run time from testng.xml without having to pass the respective parameters in the method signature.

Base Test

BaseTest (FileName – BaseTest)

Base Test is created so it could be extended and used in other tests, so we don’t repeat ourselves in setting the same configuration repeatedly. Single Base Test is created to help us set the base for Android and iOS tests.

As you can see, deviceId is captured as a part of TestNG annotation @Parameters, which will be set using the testng.xml file. Based on this field value, device configuration will be read from the “parallel.config.json” file, and accordingly, devices will be allocated to the tests.

Next, the builder design pattern is used to build the instance for driverManager and accordingly pass the deviceId values so we can run tests on the desired configurations.

Running Appium Tests

Considering the parallel tests strategy we had set earlier, the following testng.xml has been generated, which will help us run the tests in parallel in 2 Android and 2 iOS real devices, respectively.

Running the tests (FilName – testng.xml)

Check out the tag in the testng.xml file:

We have set the values for attribute as parallel=”tests” and thread-count to 4. So, once we execute this, it will start running the tests in parallel in 4 different threads.

The following table explains the devices and platform details used to perform Appium parallel testing.

Test No	Platform	Platform Version	Device Name
1	Android	11	Pixel 5
2	Android	10	Galaxy S9 Plus
3	iOS	14	iPhone 12 Pro
4	iOS	14	iPhone 11

Running Appium Parallel Tests

There are 2 ways to run the tests, which are as follows:

1. Running through testng.xml.
2. Running using Maven.

Running Appium parallel tests using testng.xml

To run the tests using testng.xml, we need to right-click on textng.xml and select
Run ‘…\testng.xml’. It should start executing the tests in parallel.

Checkout the Screenshot below from IntelliJ, which shows the execution status of the tests by running it using testng.xml:

Running Appium parallel tests using Maven

Trigger the following command on the terminal to run the tests using Maven:

Following is the screenshot from IntelliJ, which shows the execution status of the tests using Maven:

Once the tests are run successfully, we can check out the LambdaTest Dashboard and view all the video recordings, screenshots, device logs, and step-by-step granular details of the test run. Check out the Screenshots below, which will give you a fair idea of the dashboard for automated app tests.

We ran 4 tests in parallel – 2 tests on 2 different Android devices and 2 tests on 2 different iOS devices.

The dashboard screenshot below correctly shows the build name we passed from the parallel config file. One important thing to mention here is you will see all the build names coming up in parallel once you run the tests. Also, you can check the device allocation and what all tests are running in parallel.

The following Screenshots show the details of the build and the tests that were run. Again, the test name, device name, platform, and respective platform version are all correctly visible for each test set from the parallel config JSON file.

In addition, this screen shows all the metrics in detail, which are very helpful from the tester’s point of view to check what test was run on which device and accordingly view the device and Appium logs.

It also has the video of the test that was run, giving a better idea about how tests were run on the device.

LambdaTest-Build-1

LambdaTest-Build-2

LambdaTest-Build-3

LambdaTest-Build-4

Conclusion

In this Appium tutorial, we discussed Appium parallel testing and its advantages and disadvantages. We also touched on some points related to understanding TestNG and how it can help us perform Appium parallel testing.

We also discussed how to run parallel tests using TestNG on LambdaTest, which is a cloud-hosted mobile device platform and how we can view the details of the test run using the LambdaTest dashboard and its detailed build view screen, which shows good metrics about the tests run with logs, video and much more data.