Master Test Writing With GitHub Copilot [Testμ 2024]

GitHub Copilot is an AI-powered tool that acts like a collaborative coding partner that can significantly enhance your ability to write tests for your code. It can automatically generate code snippets, including test cases, based on plain language instructions or descriptions you provide.

In this Testμ 2024 session, Michelle Duke, Developer Advocate at GitHub, explains how GitHub Copilot can transform the way developers write tests by generating code from natural language prompts. She highlights the benefits of using Copilot for various types of tests and shares tips on effective, prompt engineering.

If you couldn’t catch all the sessions live, don’t worry! You can access the recordings at your convenience by visiting the LambdaTest YouTube Channel.

An Overview of Test Pyramid

Michelle started the session by gauging the audience’s feelings about writing tests—whether they enjoy it, dread it, or are somewhere in between. Michelle acknowledged that while some software developers relish writing tests, others prefer to dive straight into coding, often sidelining the testing process.

However, she emphasized the critical role of testing, pivoting to introduce the concept of the testing pyramid, also known as the testing hierarchy.

She broke down the testing pyramid, starting at the base with unit tests. These tests focus on verifying individual functions or methods, ensuring that each small unit of the code functions as expected. Unit tests are the most common, relatively simple to automate, and ideal for continuous integration setups.

Moving up the pyramid, she discussed integration tests, which ensure that different modules of the application interact correctly—like front-end components talking to back-end services or multiple microservices working in unison. At the top of the pyramid, she covered user interface (UI) tests, which mimic user behavior to check if the application provides the expected user experience.

Michelle explained that the pyramid shape represents the decreasing number of tests as you move up—hundreds of unit tests, fewer integration tests, and only a handful of UI tests, each serving a distinct purpose in the overall testing strategy.

Introduction to Generative AI and GitHub Copilot

Michelle then talked about Generative AI, highlighting how it differs from traditional AI and deterministic technologies. She explained that while traditional systems work on a cause-and-effect basis (do X to get Y), Generative AI operates on probabilities and can produce creative outputs based on educated guesses.

This makes it uniquely suited for tasks like generating text, code, and even art. She asked the audience if they’ve used tools like ChatGPT or DALL-E, encouraging them to share their experiences with AI-driven tools that assist in creative tasks and pair programming.

Then, Michelle discussed GitHub Copilot and how it can be a game-changer for developers, especially when it comes to coding and test writing. She highlighted that GitHub Copilot acts like an intelligent assistant, helping developers write code faster and more efficiently by suggesting code snippets and functions based on comments or the context of the code.

Michelle emphasized that Copilot isn’t just about generating code—it’s also a valuable tool for enhancing the overall development workflow, including testing. She touched on the concept of prompt engineering, explaining how providing clear and specific instructions to Copilot can yield better and more accurate code suggestions.

Similarly, when it comes to AI Automation, KaneAI by LambdaTest serves as a powerful AI test assistant designed to streamline test creation and management.

KaneAI is a smart AI test assistant, helping high-quality engineering teams through the entire testing lifecycle. From creating and debugging tests to managing complex workflows, KaneAI uses natural language to simplify the process. It enables faster, more intuitive test automation, allowing teams to focus on delivering high-quality software with reduced manual effort.

Demo: Setting Up GitHub Copilot in VS Code

Michelle began her demo by setting the stage with an overview of Copilot’s AI-driven capabilities. She explained that Copilot is designed to address some of the most common challenges in software testing, such as the time-consuming nature of test script creation, the identification of flaky tests, and the need for more robust test coverage.

Coding Language and IDEs

Michelle began by emphasizing the importance of choosing the right coding language and integrated development environments (IDEs) for software development and testing. She explained that Copilot is versatile, supporting multiple languages such as Java, JavaScript, and Python, making it accessible to a wide range of developers. This flexibility is crucial because different projects have different requirements, and being able to work across languages without switching tools adds immense value.

Wondering what all languages does GitHub Copilot support? Well, good news! It supports all major coding languages! pic.twitter.com/gGb9SdjE6d

— LambdaTest (@lambdatesting) August 21, 2024

She also touched on the role of IDEs like Visual Studio Code and IntelliJ IDEA in boosting productivity. Michelle pointed out that Copilot integrates seamlessly with these popular IDEs, providing real-time suggestions and auto-completions and even generating entire blocks of code based on simple prompts. This feature, she noted, saves developers significant time and reduces the cognitive load, allowing them to focus on more complex aspects of their work.

Importance of Testing

After highlighting coding languages and IDEs, Michelle underscored why testing is an indispensable part of the Software Development Life Cycle (SDLC). She underscored that testing isn’t just about finding bugs—it’s about ensuring that the software performs as expected under various conditions and meets the end-user’s needs.

Michelle stressed that Copilot plays a critical role in this area by simplifying the process of writing and maintaining test cases. Whether it’s unit tests, integration tests, or end-to-end tests, Copilot’s AI-driven capabilities allow developers to generate tests effortlessly, ensuring that testing becomes a natural part of the coding process rather than an afterthought.

Types of Testing

To provide a clearer picture, Michelle delved into the different types of testing that Copilot supports. She discussed unit testing, which focuses on individual components or functions; integration testing, which checks how different parts of the system interact; and end-to-end testing, which simulates real user scenarios.

She illustrated each type with practical examples, showing how Copilot can generate relevant test scripts with minimal input from the developer. For instance, for a unit test, simply describing the function’s expected behavior in plain English was enough for Copilot to create a basic test case. This level of automation not only accelerates the testing process but also reduces the chances of human error, leading to more reliable software.

Prompt Engineering and Crafting

A significant portion of Michelle’s demo was dedicated to prompt engineering and crafting, which she described as the art of communicating effectively with AI tools like Copilot. She explained that the quality of the prompts directly impacts the usefulness of the suggestions generated by Copilot.

Michelle provided practical tips on crafting effective prompts, such as being specific, using clear language, and structuring prompts in a way that guides the AI toward the desired outcome. She demonstrated how a vague prompt could lead to less accurate suggestions, while a well-crafted prompt could result in precise and actionable code snippets or test scripts.

Michelle also touched on the concept of iterative prompting, where developers refine their prompts based on the initial output to get closer to their desired results. This iterative process, she explained, is a key skill for maximizing the benefits of using AI in coding and testing.

Copilot Commands

Another interesting aspect of Michelle’s demo was her exploration of Copilot’s command features. She showed how developers could use specific commands to instruct Copilot on what they wanted to achieve. For example, using commands like “generate a test case for login functionality” or “refactor this function to improve performance” allowed Copilot to perform complex tasks instantly.

Michelle emphasized that mastering these commands can significantly boost productivity. By learning how to communicate effectively with Copilot, developers can harness the full potential of the tool, making it an invaluable assistant rather than just another coding tool. She encouraged viewers to experiment with different commands and to make use of the extensive documentation and community resources available.

Why Is All This Important?

One of the highlights of the demo was when Michelle shared insights from Accenture’s experience with Copilot. She revealed that Accenture had integrated Copilot into their testing workflows across multiple projects, with around 450 testers actively using the tool. The results were impressive: they reported a notable reduction in the time spent on writing and maintaining test cases, along with an overall increase in testing coverage.

Michelle highlighted that Accenture’s testers found Copilot particularly useful for tackling flaky tests—a common issue in automated testing where tests produce inconsistent results. By using Copilot’s AI-driven analysis, they were able to identify patterns that contributed to test flakiness and take corrective actions, leading to more stable test suites.

Michelle wrapped up the demo by discussing how Copilot is reshaping developer productivity, supported by some compelling statistics. She mentioned that developers using Copilot have seen productivity boosts thanks to the tool’s ability to handle routine coding tasks, suggest improvements, and generate test scripts quickly.

She shared that in Accenture’s case, this boost translated into significant cost savings and faster project delivery times. Michelle noted that while Copilot is not a replacement for human judgment and creativity, it serves as a powerful augmentation tool that allows developers to work smarter and more efficiently.

Q&A Session!

Here are some of the questions that Michelle answered at the end of the session:

1. When would you NOT suggest Copilot to give us a better idea of where and when we can freely use it without much consequence?

Michelle: I wouldn’t suggest using GitHub Copilot without a clear plan, especially for important or complex projects. It’s a tool that works best when you know what you want to achieve and have thought through the steps. Instead of jumping straight in, take a step back, involve your team, and decide how to use Copilot as part of your overall strategy rather than letting it dictate your approach.

2. Is there a command for a descriptive method/variable/etc? Naming for those of us anti-comment folks who prefer the code to describe itself where possible?

Michelle: There isn’t a specific command for naming methods, variables, or other elements descriptively, but you can use GitHub Copilot’s chat feature to help with this. In earlier demos, I used a lot of comments because that was my style at the time and before Copilot chat was available.

Now, with the chat and inline chat options, you can interact directly with the AI to craft your code more descriptively without needing to insert as many comments. It’s all about using the chat feature to guide the AI toward creating code that’s clear and self-explanatory.

If you have more questions, please feel free to drop them off at the LambdaTest Community.