A $1B Retail Giant Relies On LambdaTest To Scale Confidently And 2x Faster

Overcoming blockers with the existing vendor

In 2017, the organization posed a challenge that was also faced by many brick-and-mortar retailers. The challenge was regarding meeting the demands of modern tech-savvy consumers with the legacy infrastructure. And that was where the company started its digital transformation journey.

The company planned to move from a monolithic (IBM Web Sphere commerce platform) to a microservice-based architecture to give a faster digital experience to all of its customers in a more optimal manner. While going through the transition, the organization initially had a challenge in releasing the builds, which was happening once every two to four weeks. However, using the AWS components, they could release more frequently, up to 20 times a week, a number better than before, but not what they wanted to settle with.

Prior to using LambdaTest for testing, they were using a cloud infrastructure, including some real devices for testing. There were fundamental issues and challenges that their team was looking to overcome, which were proving to be a blocker to their pipeline while running the tests.

According to their QA Lead, “The turnaround time and the level of support was pretty poor with other providers. It impacted the pipeline while running the tests. Moreover, the test results were shoddy and took more time than expected.”

The turnaround time was too much for their team, as the existing vendor didn’t respond aggressively or proactively to solve the problem. They were testing on few real devices with a contract in place. However, the performance was so poor that there was a lower ROI (Return on Investment). These things were really important to them, as they planned to scale to stay ahead of their competition.

As they were not able to resolve those issues, they started looking for an alternative solution. The organization was ready to take a step back to switch from a real device cloud to LambdaTest, provided that they get the necessary support on a higher priority.

“We don't run thousands of tests a day, but the tests we do run are very valuable, and we need them to run quickly, and if we do have an issue, we need that issue resolved quickly, and these are the metrics that we are really looking forward to.” - QA Lead

He further went on adding - “We were looking for a partner, who can support and understand our ways of working, especially how we have changed in the last 12-24 months towards Continuous Delivery and Deployment and can give us proactive support from a Customer Service perspective. That's where LambdaTest came into the picture.”

Getting started with LambdaTest for UI testing and automation

When partnering with LambdaTest, they were looking into an immediate solution to the existing challenges. They also expected LambdaTest to raise issues that were not too significant but could help them down the line to accelerate the process of scaling up.

“When we did the POC with LambdaTest, there were two big factors we were looking into; the first thing was how things would spin up in terms of speed to enable our test to run, and the other one was very much around the support we would get.”- QA Lead.

Their team started using Selenium Grid for automation testing, along with performing tests on different browsers and devices to realize real-time testing. With LambdaTest, they didn’t have to host their own Grid, which is otherwise a costly and non-scalable task. Cloud-based Selenium Grid by LambdaTest helped save the time & resources that would have been otherwise utilized to add & maintain browsers & operating systems in an on-premise infrastructure setup.

Also, the organization didn’t have any device lab when they started focusing on the QA. They knew that they needed to have a platform that offers multiple browsers and devices to test as they grow. Setting up an in-house device farm was not a financially feasible solution. However, with LambdaTest, they are able to do all these without any hassle.

“With LambdaTest, we have experienced improved performance and also the advantage of having most of the third-party tools already integrated with the platform.” - QA Lead

More than the platform, it’s the support and response time that's helping their team. The new features that keep coming up are another valuable addition to them.

“It’s really a great platform with so much convenience, which has made functional UI testing much easier than we thought. Moving forward, as the team is more confident, we will be performing Functional UI testing more frequently and seamlessly.” - QA Lead

According to their Principal Quality Engineer, he keeps on assessing UI testing and browser coverage on both desktop and mobile devices with the team and how they are leveraging the LambdaTest platform:

• Node Js with React for Functional UI automation testing.
• JavaScript and Typescript for the front end.
• TestCafe for automation.

However, they were facing challenges around microservices, as things were becoming complicated because of those changes. They needed to make sure that if someone is working on a particular service, it needs to be there in their pipeline, like Jenkins and GitLab. They also performed consumer contract testing using pact broker, an open-source tool that required the consumer to deploy, administer and host himself/herself. It helped in sharing consumer-driven contracts and verification results.

“The QA team focuses on the scenarios where we need to carry end-to-end tests and identify the areas where UI tests can be more important. We perform integration testing for microservices and also look forward to better ways of doing that.” - Principal Quality Engineer

LambdaTest provided a much more robust solution that enabled them to test across a wide range of browsers and devices while meeting their delivery timelines. Moreover, it integrates really well and easily with tools that they were using, like TestCafe and Jenkins.

“By partnering with LambdaTest, we get quick responses to the questions, and 90 percent of the issues are resolved within a couple of hours.” - QA Lead

They now release 40 times a week and about 200 times a month on some of their key pipelines, with an average production deployment duration of around 30 minutes.

Confidence to look into the future as they scale

Being a Europe-based company, the organization’s customer base is very interesting, as they don’t have too many versions of websites or applications. However, while planning to scale, they had to consider different regions of Europe that would change the landscape of their website, with customers using different browsers from different regions of Europe. This was going to change the landscape of testing as the browser coverage would have to be expanded as well.

With LambdaTest, they got that flexibility to pay and use as per the requirement. Moreover, you could easily scale-up and scale-down depending on the requirement, making things more economical.

“I don’t have to pay for 15 parallel sessions if I really don’t use it. This collaboration has really paid off for us, and we look forward to the future add ons on the platform, which looks well aligned with our roadmap. Having LambdaTest gives us the confidence to look into that in the future as we scale.” - QA Lead

LambdaTest promises to bring a better and existing user experience down the line this year and few other things to align with their roadmap. Performance testing is one such enhancement. LambdaTest’s integration with Lighthouse can really help the organization to overcome performance testing challenges too. So, performance testing is something LambdaTest is targeting in the next six months, along with some other features that the organization has demanded.

“We have an understanding of Lighthouse and are keen to leverage that having a hands-on it. For us, page speed is really important, and monitoring page speed on different browsers will be a benefit for us.” - QA Lead.