Enabling Agile Software Testing With Automation

Over the years, I’ve worked with teams at varying levels of Agile maturity. Some, completely new to Agile, others ‘doing’ Agile (talking the talk), and very few being Agile (walking the walk). Regardless of a team’s Agile maturity, there is almost always one commonality: a struggle for QA to keep up with the lightning-fast pace of Agile software development.

When teams are learning how to operate with agility, they see how fast things move. This is a great benefit of Agile, but it comes with some assumptions that, if ignored, can cause a team to struggle with frequently delivered iterations of their product. If you’re looking to improve your Agile interview skills, check out our curated list of Agile interview questions and answers.
I consider automated testing to be an unspoken necessity for a team to truly experience the benefits of agility. A team can only go as fast as the tools, pipelines, and processes that enable it.

“Automated testing decreases regression time and increases confidence in each release.”

If you’re new to Selenium and wondering what it is then we recommend checking out our guide – What is Selenium?

Read: Why Selenium Automation Testing In Production Is Pivotal For Your Next Release?

Let’s talk about approaches that teams can take to level up their automated testing.

The realm of test automation tools and frameworks can feel overwhelming. Every team has different needs, so it can be a daunting task to build up an automation solution from scratch.

Even teams with an established automation suite will constantly find new frameworks and automation trends to follow in the space (all-in-one automation solutions, visual UI regression tools, and Selenium alternatives – to name a few!).

That said, there are generally similar approaches to leveling up a team’s automation, depending on the team’s current state.

In true Agile fashion, we’ll identify some team personas that necessitate uniquely tailored solutions to becoming automation superstars.

• Persona 1: Team has no automated testing in place; all test activities are performed with a manual approach.
• Persona 2: Team has automated testing in place, but the automation is not benefiting the team.
• Persona 3: Team has automated testing in place; the automation provides value and acts as an enabler for the team. This looks different for everyone!

Now, let’s deep dive into each of these types of teams and identify how to help them succeed in their automation journey:

Persona 1: How To Get Started If Your Team Has No Automation In Place

Identify the skill set of the team:

QA teams have different make-ups. Some are 100% manual testers. Some are 100% developers, maybe with zero testing experience. Others have a combination of manual and technical testing experience.

When working with an existing team, it’s important to know your team’s background and skill sets so that their strengths & skills can be leveraged in the best possible manner.

If an existing team’s QA members are solely manual testers with no technical background, they’ll need a different introduction to automation than developers or SDETs (Software Development Engineers In Test).

They might be interested in coding and should therefore be given proper programming training before delving into automation. This will ensure their long-term success.

On the other hand, a manual tester may not want to code but can still contribute to automated testing activities. In this case, the team may want to consider an automation implementation using BDD (Behavior Driven Development) frameworks like Cucumber so that non-technical team members can chip in and help write test scenarios.

Run Selenium, Cypress, and Appium tests on LambdaTest to scale with the demand of your website and web apps.

Automation should be used to boost efficiency in manual testing:

Examples include using Postman’s test scripts while manually testing APIs, scalable cross browser testing tools like LambdaTest, creating JavaScript bookmarklets for easier form-filling, or custom bashing scripts to generate test data. Automation engineers and developers can utilize automation in creative ways to speed up manual testing!

For a team with a mix of manual and technical testers, I highly recommend using the philosophy of paired programming and partnering up with people on the team that have complementary skills.

If you have someone with a lot of domain knowledge who doesn’t have a coding background and another person with strong technical skills but less subject matter expertise, these two could be an extremely powerful pair!

In any case, the team must take inventory of their skills when laying the foundation for their automation solution. Once this is established, there are other factors to consider before jumping into automation.

There are too many to list (and might vary from team to team or project to project), but here are a few common ones:

• Tech stack: If the technical team members solely program in JavaScript, you likely wouldn’t choose a Python-specific automation framework.
• Technical skill sets: If non-technical team members want to contribute to automated testing, consider a Behavior-Driven Development (BDD) automation solution.
• Application functionality: Mobile app? Web app? API? There are frameworks for everything!
• Understand what you’re automating: Lay out test scenarios, identify critical functionality of the application, user flows – this could help you choose the best-suited test automation tools. For example, if your app has graphs, charts, complex diagrams, etc., that are critical pieces of functionality, you might consider a solution that supports visual UI testing.

The key here is to dedicate time to planning the solution before jumping in. Involving developers and technical leads is beneficial.

Once the team has laid out the groundwork, you’re ready to start automating. I highly recommend becoming familiar with the Test Automation Pyramid concept before you start. I’ll go over some anti-patterns and things not to do below.

Persona 2: Course-correcting To Get Your Automated Testing On The Right Path

Unfortunately, it’s very common for teams to throw together an automated test solution without taking a planned approach to it from the start. As a result, testers end up with unreliable tests in an unmaintainable suite.

In my own experience, I have joined teams that have two or more years invested into an automation framework, only to discover it is not scalable for their needs and scrap the whole thing for a new one.

While I am a proponent of switching out tool sets for new, shiny ones that enable you to work more efficiently, no one wants to discard years of hard work.

If any of these pain points sound familiar, it might be time to look at how you can improve your automation.

• Tests take a long time to execute (several hours or longer).
• Your regression tests are being run on testers’ local machines.
• Tests fail a lot for unknown reasons.
• There is no easy way to run a subset of the full test suite.
• Testers are spending hours upon hours updating tests that break when there are UI changes in the application.
• Your regression suite has hundreds of UI, end-to-end, and user story-based tests.
• Test runs are unreliable and have false positives/false failures.

I’ve experienced each of these at least once! While all of these can cause a lot of pain for a tester, there are solutions. For Agile teams, everything moves fast. The most common problem that I see teams face, however, is time. How do you balance automation work with all the other day-to-day activities going on?

Prove the potential of automation:

If QA is having a hard time getting Product Owners/BAs/leadership on board allocating time to automate, show them a proof-of-concept or demo of how much time can be saved with automating.

Automated testing creates more time for QAs to test how only humans can provide faster feedback to developers and save time during regression. All of this equates to delivering features faster.

Ensure there is visibility into the automated tests:

The entire development team (QA, Devs, Product Owners) should feel some level of ownership in the quality of the product – including automated testing.

I have found it extremely valuable to get the Product Owner’s inputs on critical user scenarios, which I then turned into automated smoke testing run against every pull request, giving the whole team higher confidence that the most important pieces of functionality were always working.

When the team knows what tests are running, they’ll chime in during new feature development and make sure there’s time to automate new tests! Automation and testing effort should also be included in the ‘definition of done’ for a story. Another great way to keep an eye on the automation is to demo framework enhancements or test runs in Sprint Reviews!

Make it useful:

I have seen teams that dedicate hundreds of hours to a huge automation suite, only to run it once a month. Automation is a time investment, and it only pays off if it’s utilized fruitfully.

Ideally, you have automated tests that run against every pull request – even if that is a small subset of smoke tests. Using CI/CD tools enables you to schedule nightly regression runs, trigger executions based on code merge, and more.

Persona 3: Your Automation Is Awesome – How You Can Make It Even Better

Even the most impressive test suites can always be improved! On teams with a mature automation setup, here are a few things I like to tackle with them:

• Cleaning up spec files and focusing on improving the maintainability of the entire automation framework.
• Identifying any areas for flakiness and making them more reliable – every test suite has some weak spots!
• Looking for opportunities to move tests closer to the application code by utilizing API calls or database interactions to take steps out of brittle UI tests.
• If the QA team comprises dedicated automation engineers and dedicated manual testers, blend these roles so that there are no bottlenecks in the testing process.
• Running tests in parallel (be cautious with shared test environments, test accounts, or test data) using a tool like LambdaTest to speed up regression execution exponentially.

Continuously dedicate time to automation:

Do not let automation become ‘set and forget.’ Automation can quickly become outdated and obsolete if the team is not actively maintaining it. When developing new features, look closely at how to ensure test coverage while keeping the maintainability of the test suite intact.

Shift the testing left:

Run automated tests against every PR, after every merge request, and against every release build. The earlier in the development cycle that an issue is caught, the less costly it is to fix.

Read How “Shift-Left” Testing Can Help Your Product Quality?

Always explore new things:

Have you heard of Contract testing? What about Visual UI testing? Automation tools and frameworks are evolving and bringing us innovative ways to speed up testing as we’ve never seen before. It’s up to us, as engineers, to use them to solve problems for the betterment of our teams.

Wrapping it Up!

On Agile teams, it is crucial to have some form of automated testing in place. Without it, testing inevitably becomes a bottleneck to delivering features faster. Manual testing will always have its place in ensuring that software functions well for humans. No tool or software will ever perfectly replace or replicate the human eye and the critical thinking that is such a valuable component of testers’ mentalities. I challenge you to boundary test the current limits of your testing and take your software’s quality assurance to the next level. Your team will thank you for it!

Lauren Christianson

Lauren is a Lead Automation Engineer specializing in web-based application automation and Agile coaching. She maintains the Medium publication ‘Software QE’, and advises teams on QA & automation best practices. Outside of work, she enjoys biking, swimming, video games & traveling to the best foodie cities around the world.”

See author's profile