Sanity Testing: With Examples and Best Practices

OVERVIEW

Sanity testing is a type of software testing executed on the stable build of an application with minor code changes. It is also known as surface-level testing. A sanity test is performed to ensure the application works as expected after the bug fixes or changes in the code.

In the Software Development Life Cycle, software testing is an integral part of ensuring the performance of the software application. It is performed to find any bug or error which could lead to the failure of the software application. Such failure could be costly for the organization as it could not provide the expected user experience. Hence, it is suggested to check everything before the release of the software application in the market.

There are many different types or approaches to software testing, each of which values its purpose. Among those, sanity testing is a part that is executed to ensure the quality of software applications in less time. A quick check of the basic functionality of a software build is done to ensure its functionality.

In most test scenarios, sanity tests are run post-software build to validate the application's functionality with minor code changes.

A sanity check is a basic validation to ensure a claim, decision, or calculation is plausible, free from basic errors, and not based on flawed logic. It's a fundamental test of rationality across various contexts, not just in programming or software. It verifies that the proposed functionality operates as intended, validating that recent code modifications function correctly without bugs.

It is regarded as the subset of regression testing. Usually, the sanity test is done after the completion of regression testing to fix the bug and error. Hence, it aims to test a software application component to check whether it works after adding new changes. Unlike regression testing, it only finds any dependent missing software application functionality impacted by code changes.

Some of the key points on the sanity test are as follows:

• In the sanity test, testers and developers verify the commands and functions of the software.
• Executed when we needed to complete the testing in a short time.
• It is a type of non-scripted testing.
• It is a quick and short testing method that ensures that changes made are according to the user's expectations and specifications.
• Checks minor bug fixes.
• Check the basic functionality of the software.
• Can be executed when a software application undergoes a major functionality overhaul.
• Executed when a software application undergoes some crucial modification.

You may think that if we already perform end-to-end testing or execute other types of tests, which check each functionality of the software application, then why do we need sanity tests to be done? The answer to this is very simple. You should understand that every software testing method is needed as per the requirement. Likewise, sanity testing is important to check the new and crucial functionality of the software application.

Here, we will understand why we should not neglect or avoid doing sanity testing

• Save lots of time: Sanity testing determines thorough testing of software applications. In a situation when sanity testing fails, there is no need to perform any rigorous testing like end-to-end testing or regression testing. Thus, this saves huge time on testing as errors reported in a software application are quickly sent to the developers to resolve in case of low software quality.
• Quick evaluation: It takes less time to execute the test process. It is narrow-focused on specific functionality or features that require testing. Further, we do not require test scripts and cases to perform sanity testing. Instead, we use an unplanned intuitive approach to conduct a sanity test, which quickly evaluates a software application.
• Alleviate non-required effort: We can save lots of effort in the testing software application by performing sanity testing. It helps determine whether we are required to test the software any further or not. Additionally, testers are not required to report the issues as they are directly sent to developers to solve.
• Improve regression testing: It helps the testers to verify the critical error in the application post changes. Basically, regression testing involves re-executing a set of test cases to ensure the application's accurate working after the incorporated changes.

  However, with sanity testing, we can perform quick and superficial checks on the important features of software applications. Thus, testers quickly identify the major issues without checking the entire suite of test cases. Hence, the sanity test is a supplement to the retesting process.

• Find compilation and deployment issues: Let's understand this with an example. Imagine developers failed to compile the build, including all resource files, which could cause an unfriendly user interface. The development team may forget to add any deployment notes. This can lead to non-working and non-loading of the released software application in a test environment. With sanity tests, we can identify and resolve such issues quickly.
• Provide a quick state of software release: Sanity tests give the state of software release, which allows future planning of tests efficiently. For example, if the sanity test fails, the next task on the application development is postponed, and flaws in software release are fixed as a priority.

The sanity test quickly evaluates the quality of the software before its release and checks whether there is a need to conduct any further tests. But, a sanity test requires crucial understanding practically, which will help you gain greater insight into its application or use. Let's see this with three different examples:

• Technical example: In the software development process, the application comprises multiple code files, and its integration is hectic. Hence, developers develop executable files (software build) by compiling all such code files. This build is shared with the testing team to test its functionality.

  However, imagine an organization needs the software in a short period; we follow a quick testing method rather than giving a non-tested application. Here, we do a sanity test of the application.

• Non-technical example: Suppose you are developing a mobile application and encounter a payment checkout error. The testing team reported this error to the developing team to resolve it quickly. The development team fixed the error and sent it to the testing team for a fix. They now check for the application's working based on the changes made.

Further, they also check whether changes impacted the other related functionalities. Now, there is a functionality to make payments on the payment login page. Part of the sanity test involves validating the payment login page to ensure its functioning is in place.

From the above-stated example, it can be said that the sanity test only checks for the modified or defective components of the software application. It does not check end-to-end functions.

Sanity tests allow developers and testers to validate their use in the Software Development Life Cycle. In line with this, here are two critical priorities of the sanity test.

• Sanity test cases use standard functionality, delivering high software project value to enhance user experience.
• Sanity test cases are selected based on regression tests to fulfill the Software Requirement Specification (SRS.)

Apart from its set priority, the sanity test also has specific goals that the developers and testers must ensure. Some of those goals are as follows:

• To define the expected working of upcoming features.
• To save time and cost when the sanity test fails for a build.
• To ensure new updates do not lead to any changes in the existing function of the software.
• To validate the reliability of newly added features.

Now deep dive into more details on the sanity test and understand this with examples.

The sanity test is an essential component of software testing and is regarded as an initial step in the quality assessment of software applications. However, for perfect execution in the software development process, it is crucial to have sound knowledge of its characteristics.

Some of those are as follows:

• Part of regression testing: We perform a sanity test to check small components of the software application for its functions. In regression testing, we test an application after any changes are made to check for new bugs or issues associated with it.

  As part of regression testing, sanity testing is done before a full regression test is performed. It focuses only on the most critical or crucial features after the change is done in the software application. Hence, we can say that the sanity test is a subset of regression testing.

• Unscripted: The sanity test requires no pre-written test script or test cases. Instead, testers depend on their experience to validate the applications and their components to ensure functionality as per expectation.

  In other words, testers are flexible in validating different components of applications. This means that, as per their knowledge, they check the vital function of the application impacted by recent updates and changes instead of following any predefined script.

• Narrow and deep: The sanity test focuses on a limited set of complex or crucial functionalities or aspects of the tested application. However, the test of such functionality and aspects are done thoroughly.
• Minimal setup: Execution of the sanity test only requires basic data set-up and configuration.
• Pass and fail criteria: The sanity test gives pass or fails criteria, which are easily assessed.
• Repeatable: The sanity test can run in repetition with a similar outcome to confirm the correct working of the software application.
• Executed by testers: A sanity test is usually done by testers because this requires an understanding of software testing and technical expertise.

To practically implement the features mentioned above, sanity tests are required in various test environments to ensure the software application functions correctly.

The sanity test focuses on one or more important functions in the testing software applications. It needs to be performed in different scenarios. Some of those are as follows:

• In a situation when small changes in the application’s code are made.
• When new features are added and ready to integrate into the software application.
• After a number of regressions test is done, and the new build is generated.
• Post bug fixes.
• Prior to production deployment.
• Before integrating features in the software application.

However, how often should we perform a sanity test in a Software Development Life Cycle? Well, it depends on your requirement and the complexity of the software application. As we have addressed the goal and priority of the sanity test, we will now understand the frequency of performing the sanity test.

The frequency to perform sanity tests is based on the Software Requirement Specification (SRS) and software development stage. Imagine you are in the software development team aiming to develop an application that performs as per user expectations.

However, one of Statista's reports indicates that mobile application revenue will increase and reach $ 613 billion by 2025.

You are aware of the high revenue growth and highly competitive market. This necessitates testing the new software application to ensure its proper functioning and no error on its release. Thus, you should perform sanity tests during the early stages of software development to ensure it works post-new build. This needs to be continued several times as you improve or edit the code or add new features to the software application.

A sanity test is mainly executed to verify whether the added functionality does not give errors in existing features. Further, it also validates that the system is in a “sane” state for moving ahead with future testing.

As the software application becomes stable, the number of times the sanity test should be performed decreases. However, it should be done whenever you need to perform sanity tests. This is only to fulfill the goals mentioned earlier.

In a nutshell, the frequency with which sanity tests are performed is determined by the specific need of the software application, the stage of software development, and the number of time changes made to the application.

You may ask why we need to perform a sanity test for a software application. It is a crucial part of testing software applications and ensuring their quality. However, its significance is not just limited to this.

It is crucial to note that not all software testing types have benefits and importance. It also holds some limitations and drawbacks which are important to consider. Similarly, the sanity test is not an exception. It also has some challenges, which every tester and developer should know while performing. This is because such challenges do not hamper the Software Development Life Cycle.

A sanity test can be a helpful approach to get new functionality checked in less time. However, specific challenges with sanity tests cannot be ignored. Some of those challenges are:

• Limited scope: The sanity test cannot find all the potential issues in the software application. This often leads to false positives, which mean that the application seems to be working accurately, but actually, it is broken or failed in other test environments.
• Incomplete testing: The sanity test only performs a preliminary analysis of software applications.
• Time constraint: In most situations, the execution of the sanity test is done under tight time constraints. This makes it difficult to test the application correctly.
• Limited test environment: With sanity tests, covering all test cases in all test scenarios is impossible. When we say that there is a limited test environment in a sanity test, it means that we can execute only a few selected test cases on a particular subset of the software application.

The test's scope is limited to a specific area or feature of the software, and the sanity test is performed in a controlled environment, which may not represent the actual software development environment.

Solutions to overcome the challenges of the sanity test:

To address the above few challenges, you can follow the below-given approach:

• Always create a test plan and check the Software Requirement Specification (SRS) of applications before performing sanity testing.
• You should check that the sanity test is performed within the context of a test strategy. The approach to this is very easy. You can combine rigorous testing with the sanity test or include regression testing to ensure the accurate functioning of the application.
• You can document the testing process and share the identified bug or error with the development team. This will help to address the issue before the software application is released.

However, the sanity test is often used interchangeably with the smoke testing. This can create confusion in understanding the solution to overcome the challenges of the sanity tests. The reason is that without knowing the core difference between the smoke test and the sanity test, you won’t be able to perform it efficiently and address its challenges. So let us learn the core difference between sanity and smoke test from the below section.

Most of the time, we are confused between smoke and sanity tests. Smoke and sanity tests quickly check a software application's core function. They seem similar but differ in scope, objectives, and timing to perform tests within the Software Testing Life Cycle. To address this, let's dive into the clear difference between sanity and smoke tests.

Both sanity and smoke tests are crucial to develop a software application project. Which one should be executed first? Its answer is simple. The smoke test is executed first and then proceeds to the sanity test. However, their position in the Software Development Life Cycle is different.

Sanity testing takes place in the testing phase of the Software Development Life Cycle, which follows a deployment phase. Under the testing phase, it is a part of functional software testing and is executed after minor changes or bug fixes in the software build.

A sanity test is mainly done after unit, integration, and system testing. As the software application passes an initial phase of testing, a sanity test is executed to validate whether changes or fixes made in the application caused any new issues or bugs.

On completion of the sanity test, acceptance testing occurs, where the end-user tests whether it meets the Software Requirement Specification (SRS).

The sanity test is known to test the specific functions of the application. However, other aspects of software applications are tested in sanity testing. Some of those include the following:

• Basic function: The tester performs a sanity test to check the basic working features of the application as per user expectations. For example, if you want to verify the launch of the application, you should check the functionality of the user interface, like button navigation, etc. In such a test scenario, sanity testing evaluates those with accuracy.
• Data integrity: Testers verify the provided data's accuracy and retrieval in the software application.
• Security: Sanity testing checks the basic features like data encryption, user authentication, and access control of the software application.
• Compatibility: Sanity tests also check for the compatibility of the software application across different browsers, devices, and OS configurations.
• Performance: Sanity tests validate the application's performance under complex environments.
• Error handling: It refers to how the software application handles errors and exceptions during its execution. Sanity test checks for the error-handling capabilities of the software application by intentionally introducing errors or exceptions in the system and verifying that the application can handle them properly.
• Install and Uninstall: The sanity test checks whether an application can be installed. This involves ensuring a smooth installation process and how software runs after installation.

To determine the critical functionalities of the software application, a sanity test follows three steps, which are as follows:

• Identification

  In the first step, new functionality and features are identified. Additionally, while fixing bugs, new modifications presented in the code are found. This shows that we are required to determine the scope of the sanity test in terms of changes to be made in the software application.

• Evaluation

  In the next step, you have to evaluate and modify the new components of the software application to ensure their functioning is in line with the Software Requirements Specification (SRS.)

• Testing

  The final step is testing, where a small part of the newly associated components of the application is tested in-depth.

We have two approaches to practically implement the above steps: manual testing and automation testing. Choosing the type of approach depends on the requirement or specification of the application or the bugs identified.

Sanity tests can be executed with both manual and automated approaches. However, both have their specific advantages and disadvantages. Let's see each of them.

Manual Sanity Testing

In manual sanity testing, critical test cases are executed to validate the software’s stability and functionality. Such an approach is helpful in case of limited resources and time for testing, and changes done are minor.

How to perform?

The steps for a manual sanity test are mentioned below:

• Analyze the test cases to ensure the expected working of the software application.
• Create test cases that cover crucial functionality of the software to be tested.
• Evaluate the outcome of the test cases to find whether the sanity test is passed. In case the sanity test fails, it is marked as unstable, and then we are required to conduct further tests.
• The result is then shared with the development team to address the identified bug or error during the testing.
• On fixing the error or bug, a re-test of the application is done to ensure its stability.

It should be noted that a manual approach requires highly-skilled testers with good knowledge of software and its functionality.

Automated Sanity Testing

Automated sanity tests involve using automated testing tools and frameworks to verify the basic function of the application. It includes creating automated test cases that run automatically in the application every time a new build or release is developed.

You can prefer the automation method to perform sanity tests for complex applications, which demands significant testing to ensure its reliability and stability.

It is important to note that sanity tests are more often automated as they are a subset of regression tests. It helps to save time and resources in the testing process, along with maintaining the quality of the software application.

How to perform?

You can follow below given general steps to perform an automated sanity test.

• Define the software application's functionalities, which require testing.
• Now, create test cases that include the defined functionality of the software application.
• Select a test execution tool that automates the test cases created by you based on the functionality. Some available tools include Selenium, Playwright, Appium, etc.
• You need to set up a test environment with proper configuration. In other words, ensure the availability of software, hardware, and network resources when you execute a sanity test.
• Run the sanity test and observe the test results. If the test fails, analyze its actual cause and fix the issue.
• On completion of the test run, evaluate the test result to find any bugs.
• Now, you have to report those identified bugs to the developers. However, the best practice here includes detailed information in terms of screenshots and test log files.
• You can repeat the process to re-check the stability of the software application.

Performing sanity tests using automated test tools lower the working load of the testers and developers. It shortens the release time of software applications through the quick execution of sanity tests. Here are some popular automation testing tools to perform sanity tests:

• Selenium: It is an open-source automation testing tool utilized for web application testing. It supports multiple programming languages like Java, JavaScript, Python, etc.
• Cypress: It is an open-source JavaScript automation testing tool built on Mocha. Cypress utilizes unique DOM manipulation techniques and supports multiple programming languages like C#, Perl, PHP, etc.
• Playwright: It is a Node.js library for web automation. It is used to automate Firefox, Webkit, and Chromium with a single API.

The list of sanity testing tools above is just a few examples of many other options in automation tools. You should evaluate different options based on the specific needs of the applications.

However, automation tools have some limitations, which can create hurdles in performing sanity tests. For example, automation tools have limited scope as you can only test to which they are programmed. Further, issues of maintenance overhead and huge costs with automation tools demanded alternative options.

It is preferred to run sanity tests in a cloud platform. Cloud-based testing platforms like LambdaTest can offer several advantages for sanity testing in software development, including reliability and scalability. If you want to scale your digital experiences, it is recommended to perform cloud testing. Let us learn this in detail in the below-given section.

Now that you know the different ways to perform sanity tests and their advantages, let us dig into performing sanity testing on the cloud. Sanity tests in the cloud-based environment can lower infrastructure costs, give scalability to run automated tests, enable team collaboration, and provide flexibility in the testing environment.

Cloud-based continuous quality testing platforms like LambdaTest help you perform manual and automated web and mobile app testing on over 3000+ browsers, devices, and OS. With LambdaTest, you can automate your test suites using different frameworks like Selenium, Cypress, Playwright, Appium, Espresso, XCUITest, and more.

Follow the below-given steps to conduct a sanity test using LambdaTest.

Manual testing with LambdaTest

You can execute web and mobile application tests through the Real Time Testing feature of LambdaTest. Follow these steps to run real-time testing.

Here is how you perform real-time testing for your website or web application:

• Register for free and sign in to your LambdaTest account.
• From the modal box, click the Realtime Testing card.

• Enter the test URL, choose Web or Mobile tab, and select browser VERSION, OS, and RESOLUTION. Then click START.

  A cloud-based machine will be launched running real browsers and operating systems where you can perform sanity tests while leveraging features like one-click bug logging, video recording, capturing screenshots, etc.

  

Similarly, you can test mobile web and native applications on Android Emulators, iOS Simulators, and real mobile devices.

Automation Testing with LambdaTest

LambdaTest gives access to 3000+ desktop and mobile browser environments to run automated tests using different automation testing frameworks. To perform an automated sanity test, you can follow these steps.

• Register and sign in to your LambdaTest account.
• Go to Automation > Builds from the left menu bar.

• You will get two options: Demo Project or Configure Test Suite.

• To run the sanity test, select configure Test Suite and choose the preferred automation testing framework.

• Now, you can configure your tests and initiate a sanity test.

On a similar note, you also perform app test automation using frameworks like Appium, Espresso, and XCUITest.

Watch the below to learn more about performing app automation using LambdaTest.

You can also Subscribe to the LambdaTest YouTube Channel and stay updated with the latest tutorials around Selenium testing, Cypress testing, and much more.

Following the best practices, you can execute effective sanity tests and ensure the function of the software application as expected.

• Before initiating a sanity test, you should define its objective and requirement to make the test focused on the criticality of a software application.
• Whenever it is possible to automate sanity tests, you should always prefer it over a manual test to speed up the testing process.
• Always use a standardized test environment for sanity tests. For example, you can choose the standardized configuration of software, hardware, and network environment.
• Create test cases based on the criticality and risk of the application. It will help you to execute the priority test case first.
• To track the progress and address any issues identified, you should document the outcome of the sanity test. This involves recording test case outcomes, defects found, and actions to solve them.
• Sanity tests should be performed regularly to detect bugs or errors early. You should always do a sanity test after significant changes are done in the software applications.

A sanity test is an approach for testing software applications to check whether new functionality, change, or bug fix is implemented successfully or not. Its scope is narrow and focused, which allows ensuring whether your software release meets the requirement for rigorous testing or is too flawed to be tested.

Many testers and developers are confused between smoke and sanity tests. In this learning hub, we have discussed its major difference, which gives an idea of its distinct purpose. However, they both are essential in the development of software applications.

The sanity test saves cost, time, and effort by addressing certain application functionality early in the Software Development Life Cycle. Further, it ensures the stability and reliability of software applications through its quick check on critical features. Overall, sanity testing is important to maintain high-quality software applications and ensure the satisfaction of end-users.