Webinar: Digital Experience Testing: Need of the Hour for Enterprises [LambdaTest Webinar]

According to Forrester’s Best Practice Report titled ‘Optimize Your Digital Experience To Compete For Customers’- “No matter how embedded your brand is in customers’ lives today, if you stop meeting their expectations, it’s easy for them to switch to a competitor tomorrow. As the ratio of digital to non-digital customer interactions continues to increase, digital customer experiences and the data that firms collect about them will evolve in ways that will challenge and reward insights-driven, customer-led organizations.”

Given the pace of innovation and increasing customer expectations, enterprises have no option but to increase release velocity. However, to test for omnichannel experiences, QA/Dev teams not only need access to all possible configurations of devices and interfaces a user might access applications from – but also have a robust test infrastructure in place to have on-demand scalability for test execution of all types.

Today, enterprises need digital experience testing at the center of their digital transformation strategy to ensure a seamless customer experience.

We’re sure you would have many questions about digital experience testing, understanding the industry insights and trends challenges that enterprises are facing, and developing the blueprint for a successful digital experience testing strategy.

But worry not! Our special guest Diego Lo Giudice, Vice President and Principal Analyst, Forrester has teamed up with Maneesh Sharma, Chief Operating Officer, LambdaTest, to discuss and share their thoughts on what goes into digital experience testing and how enterprises can come up with the right testing strategy to make it successful.

If you missed the power-packed webinar, let us recap the event’s significant highlights together.

What is a digital business?

Diego begins the webinar by discussing Testing Market Trends in 2023+.

He explains that organizations are moving towards becoming digital organizations and ensuring they have a business model with software technology at its core. He further explains that software is an expression of business which means that if an organization can represent all policies and procedures in software, then ideally, it will be able to identify and implement the change of its software and pivot its business model. Therefore, becoming a master of how you build software, test, and deploy becomes very important for organizations.

Diego agrees that businesses will depend on tons of software, and choosing them should be crucial and strategic. He emphasizes that writing tons of software shouldn’t be the only focus organizations need to follow a more agile digital operating model.

It includes changing how we work, becoming more agile, managing and organizing different people and talents, and the need to upscale, and technology plays an important role.

Organizations must balance these three areas: organization/people, process/practices, and technology to transform into a digital organization.

How does this impact testing?

Testing has to evolve as practiced in past years to become a more effective software-driven organization. Diego shares the trends from the last two years of the most frequently used modern testing practices, which shows that its implementation is improving slowly, although manual testing still remains dominant with around 50%.

The speaker further shares an improving trend of testing structure in organizations, where testing is evolving from a centralized test center environment to a federated one, where engineers and testers work together inside product teams.

He also shares data from a general survey of developers about the level of adoption of automated tests. This data suggests that we focus more on implementing automated tests to fix the continuous delivery loop. However, it is evident that these tests still encounter challenges and face lower levels of implementation.

Diego concludes that by studying and comparing these data results, most organizations are positioned at the beginner to intermediate level in the continuous testing journey. We further need to make ongoing efforts to move the organizations to the intermediate stage and eventually to the advanced adoption level.

What can we do better?

Diego focuses on the way forward to keep modernizing, and continuous delivery is a very crucial strategy that needs to be implemented by every organization. It means implementing shift left, i.e., testing needs to start at the very beginning, from when we begin analyzing requirements to the release stage.

He further shares that continuous testing not only means implementing the technology but also means adopting modern testing practices like test-driven development, behavior-driven development, shift-left testing, shift-right testing, implementing monitoring and observability practices, focusing on more automation, CI/CD, test environments, etc.

Continuous Automation Testing Platforms are the modern technology foundation

Diego explains that continuous automation testing platforms provide the consistent collaboration and experience that modern testers require, from reporting bugs to enabling functional testing from a UI level to the capability of orchestrating test environments, managing test data, and executing tests effectively. These platforms should also be able to move the testing experience to the cloud and integrate seamlessly with the CI/CD pipeline.

Why is a test platform important?

In addition, he addresses the need for a test platform since testing requires a village of people; the test platform supports all the different personas needed for testing. From business testers, who test for business logic, to subject matter experts, all look for tools and platforms which are easier to implement and automate their respective requirements.

So what’s next?

Furthermore, Diego points out that testing is becoming smarter during automation phases and throughout all stages.

Additionally, he presents data indicating a growth of 20% in smarter testing from 2020 to 2021, and since 2021, the use of AI for automating, authoring, and optimizing tests has increased by 10%.

By giving one such example of an AI-based tool, Tester TuringBot, that helps developers and testers test faster and smarter and optimize testing for business coverage. It creates models to help optimize the test based on the resources, focusing on the most critical business areas. Additionally, it helps in eliminating redundant functional test cases.

Re-energize your testing strategy

Diego concluded by sharing some tips for organizations to re-energize their testing strategy:

• Keeping a check on how testing is organized, they need to be federated if they have a centralized structure. If it is completely distributed, it must be centralized to adopt a federated governance approach successfully.
• Updating skills and practices and reviewing partners to help modernize testing approaches.
• Testing approaches to become more continuous, and organizations need to think about how the adoption of AI can empower their testing strategy and focus on data-driven testing.
• Adoption of cloud-based testing platforms to support multi-persona testing in a highly scalable digital environment.

Our host speaker, Maneesh, adds that organizations must test better, faster, and smarter for seamless digital transformation. He further asks Diego, as an enterprise, what should be the starting point for their digital experience testing strategy.

Diego explains that the strategy for modernizing testing should be part of an overall program for modernizing software development, as testing is now crucial. It’s best to choose a value stream to focus on and partner with testers to support the journey within a significant environment. It’s important to start small, iterate and replicate, and review the toolset and available technology to support the journey. Leveraging external partners can provide an advantage, but it’s essential to formalize frameworks over time rather than waiting for everything to be ready.

Maneesh then asks about the wave of change from tools to suites to platforms and how enterprises should think about setting up a platform for testing.

Diego replied that testing is not an isolated platform but rather a part of the ongoing platform the organization is putting together for continuous software delivery. The continuous software delivery toolchain comprises 17 to 18 technologies, including requirements formalization, agile planning, work item management, build and authoring tools, and various testing tools. Diego emphasized the importance of platformization and integration to avoid integration hassles and to provide a testing platform with multiple testing capabilities that can integrate well into the continuous integration delivery suite of tools. Diego cited Forrester’s research indicating that 72% of organizations chose the platform approach versus the best of breed and that this approach aligns with the general platformization trend happening in the world.

Q&A Session:

Q. Where do you see the automation testing space heading from writing test cases to execution, particularly with the emergence of new-age authoring tools such as low code offering tools and AI-enabled codeless tools like GitHub co-pilot and Codeless Broad? And how do you think AI will impact this space?

A: The current capabilities of AI-enabled tools, such as GitHub Co-Pilot and Amazon’s Whisperer, mainly involve generating and creating unit tests. This is helpful because building unit tests doubles the cost of software development. There are also platforms like Diff Blue that can build unit tests for code given to it. As AI solutions like touring bots become more intelligent, they can optimize test execution and environments through machine learning. Touring bots will evolve to become more autonomous, assist in testing and executing test cases, and help developers build pipelines and configure tools and platforms.

Q. Do you believe AI can contribute to ensuring adequate code coverage in software testing?

A: Regarding code coverage, I’m still determining if it matters. There has always been a debate about this, and while AI can generate all possible combinations to achieve 100% code coverage, it’s unnecessary. What’s more critical is business coverage. We need to be smart about it and focus on testing high-risk areas critical to the business and the application’s users.

Thanks to technology, we can now determine which features are used the most in production. Based on that information, we can develop a testing strategy that is much smarter and more efficient. AI can connect the dots between the data in production and help us make informed decisions about what to test and leave out. Instead of focusing on code coverage, we should be more concerned with business and technical coverage of the high-risk areas.

Q. What do you think is the biggest stumbling block for testers within an organization to modernize their testing practices?

A: Generally, the main issues in modern software testing are organizations treating testing as an afterthought rather than investing in quality, skills, and partnerships. Quality is becoming increasingly important, as poor quality can impact an organization’s revenue and reputation. Evangelizing the importance of testing to executives and management can help break through these roadblocks, but an initial investment is required to modernize testing. It may also need reskilling manual testers, choosing the right partnerships, and reviewing sourcing strategies. It’s important to understand where an organization is starting from and apply a proper strategy based on that rather than trying to become a leader immediately through a maturity model.

Q. What’s your point of view regarding the business value that quality engineers would deliver, and what emphasis is on cost vs value?

Maneesh: As a consumer, the value of QE is not standalone but rather for the business and the application. When using a mobile app to order food or book a hotel through a website, if the application breaks, the consumer immediately switches to another provider. From a consumer perspective, the value of QE is the experience and whether or not the application is breaking. Businesses need to ensure they offer a good experience and maintain their brand. Therefore, the value should not be considered the value of QE alone but rather the value of the entire application.

Diego: Last year, I authored a report on essential modern application development metrics, introducing a framework. At the core of this framework lies a cluster of metrics known as business value, which is critical for a development team to measure. The outcome of this metric can be evaluated through net promoter score, revenue generation, faster customer onboarding, and increased profits for the business line. While testing is not the sole contributor to this business value, it has four leading clusters of metrics within it. Quality is one of the primary metrics leading to the outcome of business value, as poor quality can adversely affect customer experience and, consequently, business value.

Therefore, metrics such as rework and escaped bugs in production should still be measured. Similarly, efficiency is also linked to business value, as being cost-effective and faster increases it. Engagement and progress are critical leading metrics into business value, as improvement and engagement enhance the business’s overall performance. The four leading clusters: progress, engagement, quality, and efficiency, are interconnected with business value. Hence, this framework and dashboard assist organizations in managing these issues.

Q. Do you believe that manual testers and testing will become irrelevant with the increasing use of AI and coordination in software development?

A: Even with AI, manual testing is not going away ultimately. There will always be edge cases, and situations where automating the test cases might not be feasible. However, the goal should be to invert the current 80% manual testing and 20% automated testing ratio to 80% automated and 20% manual testing. The testing landscape is changing with AI, and new areas, such as accessibility testing, are becoming more automated.

Although manual testing will not completely disappear, more of it will become automated and autonomous. The shift will take time, and manual testers should learn new skills to adapt to the changing testing landscape. For example, testers can use high-level scripting and prompting techniques to instruct Turing bots on what to do, making their roles essential in the new world of testing.

Q. Can you envision merging the traditional approach of developers testing their code with the current trend of having a separate testing team, especially with the increasing availability of automation tools and platforms?

A: To clarify, developers test their code, but typically only at the unit test level. Functional testing, which tests the functionality of an application, is usually performed by a separate testing team or individual, such as a business tester or business analyst. These individuals are responsible for writing test cases based on the requirements and business functionality of the application. However, with the rise of API testing, developers can also test at the API level and orchestrate multiple APIs to test functionality. While developers can perform some functional testing, they still require guidance and direction from testing experts to ensure the testing is comprehensive and effective.

Q. What challenges does an Enterprise face in test execution as developers start testing APIs and functional applications in addition to writing code, given the increase in provisioning of test execution?

Diego: The challenge for enterprises in test execution is test and environment management. With cloud-based platforms, it’s easier to provision test environments dynamically and execute tests effectively. The challenge is to find tools that provide flexibility in where test environments are provisioned and to enable self-service provisioning. The governance and process around test environment provisioning are essential, and cloud-based platforms would allow teams to be serviced quickly and efficiently.

Maneesh: Testing is no longer a standalone activity but is integrated into agile and DevOps processes. Therefore, it is crucial to have everything stitched together in the test orchestration and execution space. Waiting for a server to be set up can clog the pipelines and slow down the whole process. As a result, a continuous, automated testing platform is essential for the overall strategy.

Q. What kind of strategy do organizations need to consider for Omni Channel testing, given that development is no longer limited to just web applications but also includes kiosks, mobile devices, tablets, and ODT streams?

A: Omnichannel testing has become a reality, and organizations need to shift their thinking towards a mobile-first approach. In the past, organizations structured teams in front-end versus back-end for either web or mobile-only applications. However, the rise of mobile devices has led to a more omnichannel world where customer experiences flow across different devices and channels. The design and thinking now start with mobile devices and then flow into the rest of the world, rather than starting with the big browser application and squeezing it onto a mobile phone.

Building something requires considering all channels customers may use, which can be determined through multi-persona studies. This approach requires testing across different channels and devices, such as web browsers, mobile devices, and IoT devices. Thus, omnichannel testing has become a critical part of the testing strategy for organizations.

As a closing remark, Diego shares that as we utilize more technologies like chatbots and AI, it’s important to remember that we still need to prioritize testing. We cannot solely rely on these technologies to do everything perfectly, as mistakes can still occur. It’s essential to review and supervise their actions to ensure they function correctly. However, these technologies can still provide immense value, so we should utilize them while keeping testing a crucial part of our processes. This is the direction we’re heading in.

Hope You Enjoyed The Webinar!

We hope you liked the webinar. In case you missed it, please find the webinar recording above. Share this webinar with anyone who wants to learn more about digital transformation and digital experience testing strategy. You can also subscribe to our newsletter Coding Jag to stay on top of everything testing and more! Stay tuned for more exciting LambdaTest Webinars.

That’s all for now, happy testing!

Sparsh Kesari

Sparsh Kesari is a software developer specializing in Full Stack Development. He is an Open Source enthusiast and has been part of various programs including GitHub Externship. He has also interned at organizations like GitHub India, RedHat, MeitY, Government of India. He is actively involved in the testing and QA community work as well. In his current position, he works as a Developer Relations Engineer at LambdaTest, exploring and contributing to the testing world.

See author's profile