Leading the Charge in Software Quality with Zero Bug Revolution

The full transcript

Kavya (Director of Product Marketing, LambdaTest) - Hi, everyone. Welcome to another exciting session of the LambdaTest XP Series. Through XP Series, we dive into a world of insights and innovation featuring renowned industry experts and business leaders in the testing and QA ecosystem. I'm your host, Kavya, Director of Product Marketing at LambdaTest, and it's a pleasure to have you with us today.

Today's session is particularly compelling as we explore the transformative concept of the zero-bug revolution in software quality. But before we dive in, let me introduce you to the guest on the show, Rupesh Garg, CEO and Chief Architect of Frugal Testing. Rupesh is a visionary in the software testing industry, known for his innovative approaches and strategic foresight.

With extensive experience and a deep understanding of both technology and business strategy, Rupesh has steered Frugal Testing to impressive heights, fostering a culture of innovation and customer-centric solutions. In today's session, Rupesh will share his insights into how the zero-bug revolution is reshaping software quality and how this approach is not just an ideal but a practical and achievable goal in modern software development.

Additionally, Rupesh will also address potential challenges, provide strategies to overcome them and ensure the successful adoption of the zero-bug methodology in your projects. So, let's jump straight into the heart of the discussion. Rupesh, why don't you tell our audience a bit about yourself and Frugal Testing?

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - Yeah, sure. Hello, everyone, myself Rupesh Garg. In terms of industry experience, I started my career in India, and then I moved to the US for higher studies, and then I worked over there, and for the last 10 years I've been working in India. And about six years back we started this company called Frugal Testing.

Initially, we were a SaaS product company, but we later on, during COVID, were turned into a services company. And so we are right now providing majorly, I would say, testing services such as functional testing, automation testing, API testing, load testing, security testing, the whole game of testing services, and a bit of dev and cloud services.

We have done work with about, I think, more than 150 clients still now in the US, India, Singapore, Dubai, and Malaysian markets. And we are about 80 people company right now based out of Hyderabad, Gachibowli, and yeah, that's all about us. I think then we can go into questions now.

Kavya (Director of Product Marketing, LambdaTest) - Thank you, Rupesh. So, starting off with the very first question, how can zero-bugs be realistic for all projects? How do we go about setting achievable goals?

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - Okay, very good. I think zero-bug revolution; I know we have, in fact, used this word everywhere or at many of our podcasts also. But the zero-bug revolution is not what we can achieve in one shot. But it's a granular journey, it's a step-by-step journey and we will have to set it up in every quarterly goals kind of thing.

And once we set it up in that one, we can definitely achieve this goal, or at least we can, I would say, not achieve it. I would say we can come down our bug leakage very drastically every quarter, and over a year timeframe or two-way timeframe, we can see significant changes; that’s why most of that is how most large companies are managing.

And that is how we are trying to implement this kind of framework also for our very small scale clients like I'm talking about large-scale companies like Facebook and Microsoft. That is how they manage, and we are trying to implement a mini version of that one for our clients; one of the things we are trying to focus on is a better cultural angle of the issue, and then number 2 is also about the tools AI tools which I will talk about it in the coming minutes also little further on this one.

Kavya (Director of Product Marketing, LambdaTest) - So thank you, Rupesh, for sharing that. To begin with, it's an interesting point about how you need to basically break it down into smaller goals, I believe, in order to sort of implement it. yeah, moving on to the next question that we have, right? What specific tools and techniques can help prevent bugs early in development?

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - Okay, very good. Basically, I would divide it into people, process, and technology. One is the people's angle of it that we should have that kind of QA engineers who are adaptable to the new technologies. We have a more collaborative mindset, and we have more sprint-driven people not in a waterfall model, which is extremely so.

And, anyway, even though we are really, I'm also realizing that right now. Even large companies or small companies, most companies are moving to the sprint model, which is almost a 2 to 4-week cycle, not a 3 to 6-month cycle, which used to be 15 years ago.

The same thing on the tool side; there are a lot of changes happening, even with the LLM model's arrival; even though we are using a lot of like even with the ChatGPT, we are able to solve a lot of problems in terms of test case generation, especially or the test data generation also.

And some of the fixes in our code bases also we are able to take help you know for that one and apart from that there are several tools which are you know which are being used in the market which I feel are definitely giving a little sort of edge over the regular testing cycle and obviously reducing our speed by reducing increase our speed by 20-30%.

Or maybe reducing the effort by 30% roughly like that so some other tools like testing we are using for functional automation, which uses AI to generate test cases and then do a heat map analysis. On top of it, Jenkins, which was being which is being used by a lot of people in the market, and we are also using some of that one, and then with the AI features of that one that is helping us to move much faster.

Then the same thing comes with Applitools; if you have very heavy websites or heavy mobile apps which are having a lot of pages; it's very difficult to test those applications in a very way. It's very difficult to test so many pages, and either you need to develop some kind of automation framework and then if we have some kind of self-filling automation framework, that is what makes it different, and we are trying.

We have looked at it Applitools is helping us to create an AI framework on top of the UI testing also. those kinds of some of those kinds of tools that really help, especially from AI implementation in the testing space, and then ChatGPT, obviously, I think, is the number one tool that is helping to reduce a lot of investor time for trivial work sort of thing. Yeah, that's it.

Kavya (Director of Product Marketing, LambdaTest) - Thank you, Rupesh. Very interesting because at LambdaTest, we've been focusing on bringing in new AI features such as AI-powered root cause analysis or auto-healing features into our existing products so that we can make it very seamless for the testing community.

Interestingly, what you shared about the techniques, such as the sprint model creating a collaborative environment and how all these tools with AI features are making up, all of them combined, are potentially making such a big impact on speed and accuracy when it comes to bug prevention.

So yeah, thank you for sharing that. Moving on to the next question, how can we address resistance to the zero-bug approach from teams used to older workflows?

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - Okay, yeah, that, like, I think that is all about people part of it. Like when we say people process and technology framework, in that one people and process part of it which we need to address. We can obviously start with a very small POC or very small subskilled part of the implementation of these tools and the self.

As a major part of the problem comes in, we need to have self-filling automation frameworks or self-filling test cases frameworks and also automated execution, which is integrated with CSC recycle. that I think can come number one is from the test management angle or managerial angle, it can come from the adoption angle, and number two, the costing angle.

They can have issues which we can address from that by taking a small scale work proving that there is ROI though there is initial upfront investment number one. The number from the on-floor engineers' angle, we need to have more adaptable engineers with a better IQ, fundamental IQ, I would say, which will help to make sure that people want to acquire new skills.

So obviously, whenever we are doing a pilot, we also need to take a team that is more adaptable and more open to change rather than taking a very small team that is very slow to change. So if we implement them, then others will also be convinced that there is a return coming on the investment for this effort, and then the rest of the subunits/subteams will follow the same rhythm; whichever we are implementing in team 1 up to team they will implement it.

Kavya (Director of Product Marketing, LambdaTest) - Awesome, thank you for highlighting how more or less a cultural change within a team can help drive improvement or, like, you know, help drive scalability at large and also help with project deliverability, speed and so on. And moving on to the next question that we have, are there any downsides or hidden costs to the zero-bug approach that you have come across so far?

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - Zero-bug approach is one of the biggest issues that comes with all these are money intensive initiatives and effort-intensive initiatives; it's some sort of like you look at automation testing initially, they take a lot of money and effort, but after few months you can see the return on that investment same thing comes with the zero-bug approach also because we are implementing lot of AI tools.

We are trying to hire maybe more expensive engineers because when we are implementing more sophisticated tools, we need those kinds of engineers who really get the point and on whom there is not that much micromanagement to be done for them to acquire that skill so in that sense those people are we need those kind of people who can help us to implement.

So, obviously, I would say more expensive teams in a direct form, and initially, the team size may also grow because you will have to have an older style of working team and then a new POC slash initiatives team. But as you implement and then also you might need to spend a bit on the part of the license also.

But after three to six months, once you have implemented all of it, you should be able to get rid of some of the effort in the older teams and then sustain with the newer effort on the newer teams only, which might be more expensive teams, but they will be faster, more adaptable and more eventually more open for new ideas like one of the ideas always comes that you how Facebook manages such a large application hardware.

Obviously, you cannot have very outdated engineers; you need to have the very best of engineers, even in your QA department, to manage that kind of infra when you are managing a couple of lag servers, you know, so it's not like managing two to five machines on the AWS farm. So, those kinds of then you need to those kinds of observability engineers those kinds of software are on top of it.

Understanding that software is on top of it also you need that IQ kind of people. So, that initial investment may be higher, but I think after over the timeframe, obviously, these are very highly returning ROI initiatives.

Kavya (Director of Product Marketing, LambdaTest) - I think, a very on point answer to the question because it also sheds light on both sides of the coin. I mean, there is initially an upfront cost, but then the benefit from a long-term perspective is much higher because overall, as you said, once you bring in the resources, the right resources, you're able to also ensure that the productivity is increased.

The kind of turn around time is also increased and so on. yeah, eventually, at the end of the day, the goal is to bring out focus on quality. So, as I said rightly, thinking from a long-term benefit is what we should focus on.

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - So, yes, we have implemented it in testing. Also, we have also implemented it in our marketing framework. Also, we have implemented a very expensive framework right now in the last two years. We implemented a very expensive framework initially, and the cost was very, very high.

I felt that other people were sort of laughing at me that, hey, you are spending so much money we don't even get why you are spending so much money, but I felt that automation is the need of the hour; you cannot even generate leads, cannot generate meetings, cannot get new business if you are working in an older style.

Like for example, if you look at the marketing field, the older style has been always cold-calling people, trying to persuade the prospect. But the new style of marketing has been more from branding, positioning, better messaging, or better like even we are doing this podcasting thing. This also can be said to be a subtle marketing tool, you know.

So to implement podcasts, to implement a lot of other initiatives in marketing also, number one, you need tools. Number two, you need more sophisticated people in your department. It cannot be done by the older style telecallers who are only doing group calling every morning.

And people are getting retweeted with their calls. So same thing we felt now we are getting RYE of our framework. It's the same thing, I feel, even for our dev teams, product management teams, or testing teams once we implemented even AI tools, some of the AI tools which are like really increasing speed it is giving us a lot of efficiency like we were doing website testing our own website.

So we were getting lost; we were saying now these pages are growing like hell we don't even know how to make a tree structure. I needed to hire one guy, and then every day the developer changed the tree structure and it was a headache then we said maybe there must be a tool that does reverse mapping of the software of the website.

And then we found a software which sort of makes a mind map of the website, and once we make a mind map every morning we can make it, and it tells you the changes which happened from the last code release so it makes our life very easy I don't need any resource in the office I can just give that hyperlink of the website to that software myself and create a tree structure out of it.

So yeah, those kind of things. For that one, that's why I was saying more sophisticated engines, obviously you need in your team. And then you need to invest some money in the tools also.

Kavya (Director of Product Marketing, LambdaTest) - Yeah, very interesting because this is the same strategy that you sort of implemented when it comes to quality engineering is what you have sort of implemented across different domains within or functions within your organization.

And I think that would be a good learning for organizations that are starting small, perhaps, and even those who are at a larger scale. I mean, because of the way the teams are structured these days, I mean, the whole purpose is for those teams to be more agile in nature, right?

So, from that sense, automation definitely hits with that. Great. And moving on to the next question, what other metrics can measure the effectiveness of the zero-bug approach on software quality?

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - Yeah, I mean, the biggest metric is always the defect leakage. That is the only metric which is the final verdict about your product or about your work done. For example, if you look at a CEO, their biggest verdict is always the top line and the profitability of the organization. Otherwise, everything else is a pretty much no-value metric for them.

The same thing for QA teams, will be, or other product teams also, will be, the biggest metric will be how many defects are getting leaked, and number two, how much effort we are burning are we more effort every year and is our effort stabilizing and the budget stabilizing every year or coming down if those kinds of metric we can measure the number of people deployed.

Number of production incidents that happened as you know, effort burned and the cost of the effort burned in terms of cost which can be in the software and the hardware angle and the human resources angle if we can manage through that one we know are we doing like how we are doing as compared to last year.

Another thing you know obviously comes in team management is also how much your QA stack is dependent on the different people for example if people are leaving is your stack will fall apart or will the stack will remain there if people leave because sometimes we have built a stack which is very very dependent on that particular person or a few engineers.

Once they leave even that stack goes away nobody else can get it so you have built a stack which is replicable which other people can get it and which other people can easily follow so those kinds of things i would say yeah i mean which we need to measure our teams output or our own output sort of.

Kavya (Director of Product Marketing, LambdaTest) - Thank you, Rupesh, that really helps. So understanding all these different multiple metrics can definitely help it. just from a high level perspective, I want to list down a few so that it becomes easier for our audience to understand. Defect leakage is one, production incidence is one, and cost of the effort is one, and the other being cost of the tools, of course, yeah. So, four of these, yeah. Great.

Thank you so much for sharing this. And moving on to the next question that we have, can you also highlight some of the real world challenges that were encountered during the zero-bug implementation? And how did you go about overcoming those challenges?

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - Okay, yeah one of the examples I can tell we did with a Singapore based client it was a fintech company where we had scalability issue for the product it was a fairly complex API product you know with the microservices we did a we the engagement was for I think about three months and we then we did we did a full performance testing and scalability exercise, and we were able to bring down the response time quite drastically almost by one fiftieth of the response time.

So what were the main things we implemented was that we took help of we not only did the front end testing but we did the API testing, load testing also. Number three, we also collected data on the backend on the bad performance SQL queries. Number four, we looked at each API and the server architecture also like is the load balance is working well?

Are we having the right memory, CPU utilized from the servers by optimizing all of that one we were able to reduce a lot of HTTP error codes you know, 50x error codes 40x error codes and we were able to pretty much move into HTTP for 20x response codes which gave it a huge you know basically the application became stable number one number two the response time became much faster so that is the one thing we implemented.

One more thing I can tell you which we have just implemented for like one of the clients which for which we had it was like a utility client where what they do is like you if you look in US or Europe people like here the metering data comes manually right the electricity bills water bill comes by hand but normal in advanced markets.

The bills don't come by hand or by email they are collecting the live dashboards and they're they're sort of sending automated emails or SMS, and you can only check the dashboard yourself by going to that particular website. So we had that kind of client which is sort of like they come in the smart home category in India and which is being implemented at very very few places right now in Indian market.

So we did that kind of automation testing for them or BI dashboard testing for those kind of clients. And then we were dealing with a telemetry database and almost up to 20 million rows very very sizable DB and then there we implemented a you know we implement Postgres SQL, also SQL, Selenium with Python with that one we were able to bring down we were able to automate the whole dashboard process and also capture the response time.

Plus check the backend data if it is stored well or not basically like if I'm managing very very large database. It's very difficult for me to understand am I having right data in my tables or how do I check, how do I make sure so that kind of framework we built so that is another example and then one more thing which I was saying about like LLM model implementation which we have initiated now in last one year through which ChatGPT creating lot of auto functional testing cases helping with the test data that is also saving lot of time.

Like I was talking about even number 3 that tool, I forgot the name of the which is doing a reverse mapping on the website. So if people are involved in every front-end simple website testing, not complex back-end products with business logic, then those kinds of tools can be very helpful, and they can really help you to find very quick bugs.

You move faster because one of the tenets of zero-bug revelation is that finding bugs very fast and very early means like sort of monkey testing in which you find bugs, which normally people call to shift left also that try to find a bug as early as possible or even by hours it's very beneficial if it's a sprint of two weeks if you find in the day 1 or 2 or 3 the bugs is one lot more can be a higher chance of result than the last day of the sprint because then anyhow all the work is already done by the dev team you can't bring changes on the last night so that way so yeah these are a few cases I can talk.

Kavya (Director of Product Marketing, LambdaTest) - These are all fantastic case studies, and congratulations to your team for the successful implementations of all those. I think this really showcases the value of how the zero-bug approach can be implemented across different industries and in different use cases, right? Especially given the real-world scenarios that you just shared with our audience. Thank you.

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - Thank you.

Kavya (Director of Product Marketing, LambdaTest) - Moving on, Rupesh, on to the last question of the day: how can organizations ensure that zero-bug philosophy becomes ingrained and sustainable in the long term?

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - Okay, yeah. I'm the ingredient one is like there need to be management or the management support for this initiative and then number two I think there needs to be some additional budget allocated maybe initially which will help them to move fast deploy that kind of things and number three the company need to be more like I've said saying what having teams which are more adaptable to these kinds of changes.

We need to find those early adopters who were normally in any new product deployment or even when any company sells a new product in the market, they find early adopters, and then they try to make sure that the product is getting adopted through them like Walkman in the 1960s was implemented through that idea only finding early adopters.

And that's how it became a rage till the point iPhone came so that’s why the scone was so so profitable at that time also. So yeah, one is the cultural angle, number two is the angle, the skills angle. Number three, like I was saying, finding early adopters or I would say, fast engineers in your team who can accept that idea. And then finding a few engineers, managers or leads who are open to those kinds of new ideas.

Then you will find the backend teams the followers; they will then you'll adopt it after six months; a few teams have already adopted this approach and have shown the results because it's not only adopting the approach; there should be some benefit also tangible benefit in the cycle time reduction and the cost reduction if that is achieved in one or two cycles or product cycles then other teams cannot say no to you they will have to implement these ideas so that's it. Yes, that's all.

Kavya (Director of Product Marketing, LambdaTest) - Thank you, Rupesh; that has been very insightful, and it is, of course, inspiring to hear how you have sort of led the zero-bug philosophy or approach and coordinated implementation in several organizations, to begin with.

And, of course, I hope that you continue on that successful journey. So, as we wrap up today's discussion, Rupesh wanted to check if you have any last words of advice that you would like to share with our audience today.

Rupesh Garg (CEO & Chief Architect, Frugal Testing) - Hmm, no, I think I've shared all my points. I don't have much fun.

Kavya (Director of Product Marketing, LambdaTest) - Awesome. Thank you so much, Rupesh, for sharing your insights. It was really interesting to hear how the zero-bug approach can transform software quality and set new standards within the industry. I really appreciate your time and expertise in sharing these valuable strategies with our audience.

And to our audience, thank you so much for joining us today. I hope you found this session insightful. Before we part, do subscribe to the LambdaTest YouTube Channel for more episodes of the LambdaTest XP Webinar Series, where we continue to bring you cutting-edge discussions from around the globe. Once again, thank you so much, Rupesh. Have a great day, everyone. Bye.