GenAI for Quality Transformation

The full transcript

Sudhir Joshi (VP of Channels & Alliances, LambdaTest) - Hello, everyone. Welcome to another exciting session of LambdaTest XP Podcast, where we bring thought leaders in the field of innovation and learn from them the best practices in the field of software testing.

I'm your host, Sudhir Joshi, heading alliances and channels at LambdaTest. In today's webinar, we'll explore how AI/ML, specifically GenAI, is shaping the field of quality engineering with our guest speaker on the show, Mallika Fernandes, an industry veteran and managing director of quality engineering at Accenture.

Mallika is a distinguished IT leader with over 25 years of industry experience. As part of her cloud first group at Accenture, she spearheaded innovation, automation, and differentiated assets in quality engineering. Her impressive portfolio includes over 15 patents for groundbreaking innovation, showcasing her deep passion for AI/ML and cloud quality transformation.

In today's session, we'll delve into the core of next-gen quality engineering and how it is transforming to pace with rapidly evolving technology. With that, I will hand over the mic to Mallika and request her to share a bit about herself and how she can inspire or help aspiring women in tech. With that, over to you, Mallika.

Mallika Fernandes (Managing Director, Quality Engineering Lead, Accenture) - Thank you, Sudhir. Well, I think you've covered my introduction, but just to add a little bit more, I'm Mallika Fernandes. I've been in Accenture for about a decade now and have been in the industry for 25 years. In fact, I spent the 15 years of my career primarily playing a delivery kind of role.

So I've interacted a lot with our clients, and I've built a lot of transformational projects and delivered them to our clients in those initial 15 years, but when I joined Accenture, I was offered a role to move to a capability. And that was more like an individual contributor at that point in time when I joined 10 years back and completely focused on innovation.

So the first thing I thought is, can I really switch or move from doing something that I was comfortable with for a long time to picking up innovation and driving assets and thought leadership. But thanks to my mentor, I accepted that role. And I think there has not been any looking back from there.

It's been an amazing 10 years that I've spent until now in this role as somebody who's been contributing towards innovation, building assets and also building the right future thinking in terms of how we can move the needle on quality engineering.

Sudhir Joshi (VP of Channels & Alliances, LambdaTest) - I mean, getting out of your comfort zone is one, probably half battle won. And then, know that having a great mentor is all you need. With that, we'll have a series of questions, and Mallika would love to learn from you.

So, of course, congratulations on the Accenture FY23 Inventor Award. I'm sure it means a lot. Can you discuss one of your patented innovations and its impact on quality engineering? That will be really helpful to understand.

Mallika Fernandes (Managing Director, Quality Engineering Lead, Accenture) - Sure, as I mentioned, I've been playing when I started this role in an individual contributor capability role. So, that platform itself was perfect and ideal for ideation and innovation. And all I needed to have, maybe, was a curious mind to think of ideas and innovate.

So one, if I have to talk about one patent, I'll select one that is very close to my heart because this is specifically on training, validating, and monitoring artificial intelligence and machine learning models that I worked on in 2016 and 2017. And we filed a patent in 2017.

Now, the focus was on the time when we were adopting AI models, and around a decade ago, you know, we were still starting to experiment with how AI can help us in testing and quality engineering.

So we were looking at how these models really bring different kinds of biases and why it is required to train them appropriately and to validate them and also to monitor them once they are in production to make sure that the machine bias is addressed to at least a good extent.

So I'll maybe give a couple of examples from the work we did at that time. For example, Google Translate, right? I mean that we all use. And now this issue is fixed. But at that time, they did have, I would say, a gender bias because when you enter something as simple as she is a doctor and he's a babysitter and translate that into a gender-neutral language like, say, Turkish.

And then translate it back into English; what you'll find is that it automatically switches to he is a doctor and she is a babysitter. Now, this kind of bias was at that point in time quite significant, and AI models needed to be trained to address this kind of bias because of a larger amount of data.

And humans have bias inherently, but it's about teaching the models to look at and create an environment that is right and that is safe for everybody to consume. And that was what the paper and the patent were all about. And yeah, that's one thing that I hold very dear because it went on to get published by MIT in their books. there was a chapter that was titled Want the Best Results from AI? Ask a Human.

And they published this book in August 2019. the book was called Why Humans Matter More Than Ever. So the whole focus at that point in time was bringing that human intervention. And there was a lot of global research happening on this topic. that's why I hold this patent very close to me.

Sudhir Joshi (VP of Channels & Alliances, LambdaTest) - Very nice, and congratulations on having this published in the MIT Book. Very interesting. Maybe a follow-up question, Mallika. How much of this, as a practitioner, have you been able to implement in real-world projects?

Mean, of course, this is an excellent example of gender bias in an AI model. But beyond that, how did it help Accenture as a business to kind of, you know, implement in a real-world project, real-life project?

Mallika Fernandes (Managing Director, Quality Engineering Lead, Accenture) - So most of whatever research that we do or I do is almost always translated into an asset or an accelerator or a point of view or a framework that we can take to our clients. So the focus is always on looking at topics that are very relevant to our clients, and that can impact the improvement or the efficiency of the work that we're doing for our clients, and this one was no exception.

We went on to build a good framework and a tool that will help teams with some of the the algorithms, machine learning algorithms. And we went on to implement the tool in several clients. In fact, even today, with the advent of GenAI, responsible AI is a very significant topic.

And my colleagues in Accenture are working on that topic, you know, quite a bit, but yes, I mean, just to answer your question, yes, we did go on, we built an asset. It was used across a number of clients, and we saw good success, not only at that time, but we continue to do even now as well.

Sudhir Joshi (VP of Channels & Alliances, LambdaTest) - Super. I think my buildup question on that is we all know quality engineering is transforming rapidly. I was referring to a report from Evendes QA 1.0, QE 2.0, and QE 3.0, which is all about automation and kind of packaging solutions around digital transformation, right, you know, in terms of no code, local tools, continuous testing.

What key changes have you implemented and mentioned, you know, in terms of GenAI, but is there anything beyond or with GenAI at Accenture to stay ahead of the competition?

Mallika Fernandes (Managing Director, Quality Engineering Lead, Accenture) - Well, that's a great question. And I think it is very pertinent to what we're looking at today. But I think the focus is that technology itself is changing so rapidly, right? That, I mean, we had the advent of cloud about four, five years ago.

So, the focus was on how we could look at quality engineering for the cloud. And then now you have GenAI. So I think the whole technology is changing so rapidly that you also have to keep aligned and build next-generation quality engineering that is aligned with transforming technology.

And that's what we always strive to do here. So when I look at how we do or what we can do to help our clients stay ahead, It's about providing a very holistic view of next-gen quality engineering, which includes elements of modernization.

And I find that a lot of our clients are moving from wanting to modernize the landscape, they're moving to the cloud as well. So there's a lot of migration work as well that is happening and also building cloud-native services from the bottom up.

So, in all these cases, bringing together a very holistic end-to-end testing strategy is what is important, and when I say end-to-end, it would include elements of functional testing and automation, also include the performance side of things, both functional and non-functional performance testing, resilience testing security testing.

So that is what I call an end-to-end test strategy, which is a combination of several different elements that also look architecture layer, look at the data layer, and ideally look at the whole end-to-end system that is available. And that's what we need to think of building a very holistic test strategy around.

It's also think for us to stay ahead; it's not only what we do internally where we are building the right assets. So we do focus on building assets ourselves where we are using AI/ML to improve the efficiency of what we do for our clients, but we also embrace partnerships because there is so much innovation happening out there in terms of automation that our partners in a way are bringing a lot of new ideas to the table.

So it's about including tools that are coming in from our partners and augmenting that with innovation and ideas that we can build internally to bring the best that is possible to our clients.

Sudhir Joshi (VP of Channels & Alliances, LambdaTest) - That's a great thought. And you spoke of modernization, know, I'm sure there are many initiatives, but one I recall at the top of my head is Rise with SAP. With the whole legacy on-prem-based solution now moving to the cloud, it's a massive migration effort.

And we look forward to some innovative solutions around helping this whole transformation and accelerating something coming up from Accenture. With that, I think this is probably quite a person. I have had a larger team here at LambdaTest in the past as well.

This is something I would love to learn from you as a leader and as a role model to many taking up QE as a profession. What unique challenges do you think you can share with these QE professionals they may face and what they should from principles or how can they overcome and become a leader like you, Mallika?

Mallika Fernandes (Managing Director, Quality Engineering Lead, Accenture) - Well, think just going back a little to the previous question that you asked, right? So when you look at, I think, changing technology, focusing on learning, being open to changing your mindset, I think that is the most crucial thing that one has to really keep in mind because technology is changing so fast.

It's not just about understanding a technology, a coding language, or a particular tool. It's about being open to adopt and to accept and learn new things as technology is changing. So I think the key thing really, and what has helped me, is to be staying afloat with the news that is happening and keeping an open mindset, and building a culture of innovation within the span of your team as well.

So, because usually when you look at innovation or when you look at tech challenges, it's not solved by a single person, right? It's not a single entity. You need to build a team of people who can bring you those fertile ideas that can mature to become the right assets and tools that can help our clients.

So I think that is one of the core things just having an open mindset, and I'd like Steve Jobs here when he says, know, stay hungry, stay foolish. So I think building skills and confidence is what really should keep you ticking. And another key thing that really helps me is following some kind of passion that you would have in life.

And for me, it's about fitness, so I focus a lot on fitness. So, I start my day with an hour of fitness every day. And that brings about that positive energy throughout the day. So my advice would really be to take time out to do something for yourself. Value yourself so that you can give back to your job or to what you can give back to your organization and your clients.

But it's really about taking that time off for yourself and having that balance in terms of work and life and spending the time that you need to reinvigorate your thinking and your mindset. And the best ideas always come when I'm out; maybe I'm passionate about gardening as well.

So when I'm out in my garden, that's when I sometimes get good ideas and I'd like to have a paper and a pen out there so that I can jot down something quickly. I think taking the time for yourself is another very important thing to focus on. And the last thing I would say is networking and mentorship.

Building your brand, being visible, and opting for leadership training possibly, which will help you build your brand. I think that is another key area, networking and mentorship that one has to be focused on. especially for women, I think we always fall prey to something that I would call imposter syndrome, right?

Where we doubt our own accomplishments. I think that is a feeling that you sometimes get that you don't belong or you doubt whether what you're thinking is the best, and you always want to do the very best before taking it out there. So I think fostering that kind of a culture of recognition, of positive feedback, of reinforcement is really the key to ensuring that you're building that team of innovative minds.

Sudhir Joshi (VP of Channels & Alliances, LambdaTest) - I mean, I can't agree more. I really love that of all, mean, I really love that you take good care of yourself. So what we talk about here at LambdaTest is, when it comes to setting up the priorities, first you yourself, take good care of yourself, then it's your family, and then everything else, right?

Because if these two things are sorted, you can now take care to do the best of your performance in other parts of the world. But very, very interesting. I think what we learned from you is, of course, how you can become better and all the tech things.

But actually, a company of size like Accenture, we see a huge, I mean, and this is my just experience, they're always in inertia towards change. There's a bit of discomfort and since you're bringing so much of, you've got 15+ patents, it means you have to convince people, you have to convince them to kind of think about this innovation and especially specifically in quality engineering.

My other question would be, what strategy have you taken and what do you suggest other fellow inventors to kind of pick and track and have a success like this?

Mallika Fernandes (Managing Director, Quality Engineering Lead, Accenture) - So I think, well, the first thing is to be updated with industry trends. So keep your mind open and be focused on accepting ideas, however small, because it's usually not one big grand idea that gets implemented, right? So when you look at patenting or when you look at innovation, it's that small spark of an idea that could actually turn out to be patentable.

So focus on any aspect that is innovative, any aspect, however small, and build on top of that because it's finally, you know, and I'll give you an example here. When we were looking at how to build quality engineering for the cloud, there were these aspects like we're using serverless architectures; we're using containers, which was a little different from the way we looked at on-prem for a long.

So the idea of how can you look at, say serverless test automation when you don't have a UI, that was something which finally became a significant topic, right? But something as small as just looking at an architecture and saying, here I have this serverless architecture.

Has anyone thought about how to automate it, or am I containerizing my applications, but have I thought of the best strategy to build to test these containers, which may be very different from the way I'm thinking about doing my functional testing?

So topics like these, which are diverse, are, I think, important to pick up and build on for us to innovate as well. So it's not just building a big tool or a big platform, but it's about picking every small idea that really matters. And I think that's the main thing.

And the second is that when we talk about quality engineering, we are not limited always to just doing manual testing. So thinking out of the box when it comes to automation and bringing new ways to automate, for example, now using, know, GenAI, I'm sure no conversation is complete without bringing GenAI, but thinking of evaluating your large language models itself.

How do you automate the evaluation of models? I'll go back to my first patent that I spoke about, where we looked at how do you evaluate and validate AI models. So, thinking out of the box for these different new technologies is important and also encourages experimentation and risk-taking because not every idea translates into being something innovative, whether it is an asset or whether it is a patent investing.

I mean, take some risks so that, know, not an, and even for me, it's like not everything we've thought about has become successful. There have been some areas of thought process which have not worked out, which have, you know, I would say which, which are failures, it's only building on those failures that you get the confidence and you also face success.

And I'd also like to quote this. It's this phrase that I've heard. It's not something which I coined, but something called aligned autonomy. I think providing that aligned autonomy to your team. And I heard this being used at one of the events I attended.

But I really liked it because I associate with it where you have your team aligned to the vision, the larger vision that you have set for the organization or for your team. However, they have the autonomy to do what they want. So they can think, they can ideate, but the vision is aligned with autonomy.

So that's another key takeaway that I did from one of the sessions I attended and I'd like to share that as well. But then the bottom line would always be to stay focused on your clients. So, whatever you do, stay focused on your clients. That's the mantra that we've been adopting.

Sudhir Joshi (VP of Channels & Alliances, LambdaTest) - Superb. With that, I have a last and most interesting question for you, what do you think GenAI would be a game changer, or is it still a buzz, especially, you know, in the QE world?

Mallika Fernandes (Managing Director, Quality Engineering Lead, Accenture) - Well, yeah, that's the question I was hoping would come up because it's my current focus as well. I think GenAI has tremendous potential in the area of QE, primarily because firstly, we generate so many artifacts, right?

Large language models can help us to generate very easily because one of the key use cases is the generation of various, whether it's your test cases, your test scenarios, your test data, or your automation scripts. I think GDI is a powerful tool that can be adopted for quality engineering.

And I look at it in two ways. The first is using it for automating areas that we could never automate in the past, like manually writing test cases and scenarios. Now you have a tool that can help you there. So there is this exponential jump in terms of efficiency that you're getting. The second area is areas that you could already automate, like automation scripts.

However, that also now gets a huge impetus because you can do it faster, you can do it with a lesser amount of data. these are the two genres that I primarily look at, but it's really, I think, helping us in areas beyond the generation as well because it's also helping us in synthesizing data, in looking at reports, in providing a very concise way of consuming information, and reporting, like let's say test coverage, understanding what amount of coverage is actually there until now, and how you can build on top of that.

And finally, prediction algorithms have always been around, but with GenAI, it's again a big leap in terms of what and how you can predict as well, because now you have more amount of data that can be consumed easily. And I think the biggest benefit is the availability of a lot of open-source models and new models that are coming every day.

Because when we started working on this about nine or ten months ago, there was only GPT 3.5 that we started using. But just it's less than a year, but the number of models you've seen an explosion of LLMs. You have GPT 4.0 now, and you know, for example, Meta has Llama, you have Gemini from Google.

So the amount and availability of these not only large language models but also small language models as well, which have come out in the market, can help you to build very niche and customized solutions for the industry.

So you can have that industry knowledge. the problems that most of our clients have about data and training models and information going out and security issues can also be contained with LLMs because of the ragged way of working.

So, with retrieval augmented generation, you don't have to train the models. The data is safe. It can be used on the fly and be used to generate a very powerful set of assets and output. So I'll say GenAI is truly that moment in AI because I've been looking at this for 10 years now, which is like the shining bullet. And I hope to see it just improve from here.

Sudhir Joshi (VP of Channels & Alliances, LambdaTest) - Super, very, very helpful. And it's time to wrap up our today's session. Mallika, thank you so much for sharing valuable insights. And to our listeners, thank you so much for joining us today.

Stay tuned for more such insightful episodes from our thought leaders in our XP Podcast Series. Thank you once again, Mallika, for joining us today. It's been a pleasure hosting you. Have a good day.

Mallika Fernandes (Managing Director, Quality Engineering Lead, Accenture) - Thank you, Sudhir, for the opportunity, and have a great day as well.

Sudhir Joshi (VP of Channels & Alliances, LambdaTest) - Bye-bye.