Continuous Testing Using Shift Left Testing Approach

Software Industry is changing at a very rapid pace and ‘Time to Market’ has become more critical than ever before.‘Waterfall Model’ was one of the most prominent software models that were used by organizations, but it had a major drawback – testing was done at every stage of software development due to which highly critical bugs were left un-noticed till the software reached the production stage. Such a scenario could also lead to business losses, in case ‘buggy software’ is shipped to the end-customer, a problem that is present in every traditional ‘develop-first-then-test’ models. This gave rise to more evolving models like ‘Shift-Left testing’ & ‘Continuous Delivery testing’ where testing is considered an integral part of the development function.

What Is Shift-Left Testing?

In a traditional development model, testing is performed only at the end of the development cycle i.e. Requirement gathering (Planning), Analysis, Design, Development, Testing, and Deployment. There is a cost associated with testing, identifying & categorizing issues, and pushing the development changes into the product. The costs might rise in case the test/verification team comes across an ‘issue of very high severity’ (release blocker issue) at the end of the life-cycle. Such issues could even result to stall the entire release.

With Shift-Left testing, testing is no longer performed in isolation by the test/verification team; but it is performed in sync with the product development. With such an approach, the testing teams are more empowered since they work in close collaboration with other stakeholders of the project i.e. product planning team, project planning team, development team, release team, etc. As testing is tightly coupled with the development process; bugs can be identified & resolved at a much faster pace. Though the development & test team work in sync from the beginning of the project, a transition to DevOps (Developer + Operations) is equally important using the Shift-Right approach. The combined approach (of Shift-Left testing & Shift-Right testing) is termed ‘Continuous Testing’.

Continuous Testing

As mentioned earlier, Shift-Left testing is more about performing the development & testing job in more synchronization; on the other hand, Continuous Testing synchronizes testing with Development & Ops i.e. DevOps in order to achieve the necessary development & business goals. Continuous testing brings the best practices of Shift Left testing by integrating DevOps as well.

Quality Assurance (QA) team plays a vital role in the process of Continuous Delivery, a process by which new changes are released to the end-customer in a more sustainable manner. Their job in Continuous Delivery is to accelerate the deployment by ensuring that bugs are solved by the development team & changes are pushed to the relevant branch such as production environment or staging environment. Continuous testing ensures that bugs unearthed & fixed at the right time so that a good-quality product is shipped to the end customer.

In simpler terms, Shift-Left testing stresses on the importance of testing at early stages of the SDLC (Software Development Life Cycle) i.e. integration of QA with Dev; Continuous testing adds the factor of DevOps to the overall equation. This valuable addition makes it possible to get issues fixed & ship changes to the end-customer at a faster pace.

Differences Between Continuous Testing and Shift-Left Testing

Now that we are aware of Shift-Left testing and Continuous testing, let’s have a look at some of the key differences between the two

1. Testing beyond UI: In Shift-Left testing, there is an increased amount of focus on UI & UX testing before the release. However, a web application or website is much more than impressive UI. Along with effective UI, performance also plays an important role in shaping up the product. As a developer, you have to ensure that transition between the pages is smooth/glitch free, the right mix of web & local components i.e. hybrid approach is used, CPU consumption is minimal, Thermal requirements are met, etc. These factors which go beyond the ‘look & feel’ of a website/web application are taken into consideration during Continuous testing. Hence, Continuous testing is more optimal since it takes into account ‘UI, UX, and performance metrics’ which are the key elements for any kind of product (website/web application)
2. More Real-time scenarios in testing: It is a known fact that bugs can occur at any stage of product development. This makes ‘product testing’ an integral part of the development life-cycle. Shift-Left testing focusses on testing at early stages i.e. pre-production stage of development, whereas Continuous testing focusses on pre-production to post-production part of the development cycle. There are some bugs that occur only when your system is being tested under extreme conditions and the possibility of unearthing such kind of issues at an early stage is unlikely since the ‘components/modules’ are still under design/development.
   
   This is where Continuous testing can play a pivotal role in performing thorough testing of the product. Consider a mobile application (app); the app might perform well when it is executing under ‘the ideal conditions’ i.e. CPU cores are executing at the highest frequency, GPU is executing at a high frequency, etc.; whereas the performance would drastically vary if the testing is performed on mobile devices that have low configuration. Along with the performance, the look & feel of a web-app might vary from one device to another e.g. the manner in which a ‘page/component’ of a web-app is rendered & displayed on a ‘mobile device’ that has Android-O could be significantly different when the same page is being rendered on a ‘tablet device’ that has Android-O. There is a huge amount of fragmentation as far as mobile & tablet devices are concerned and it becomes more complicated with the different versions of OS’s that exist for the devices. In order to ensure that your web-app works perfectly fine on the designated devices and respective browsers, a thorough cross browser testing has to be performed; something that cannot be done on a web-app which is still in the pre-production phase. This is where Continuous testing plays a pivotal role since it takes real-time scenarios into consideration.
   
3. Increased learnings, more pro-activeness: Shift-Left testing empowers the testing/verification team since it provides an opportunity to the testers to work in close collaboration with the product managers, project managers, developers, other test engineers, etc. This helps in inculcating a ‘development mindset’ within the test team. Such an approach helps them in visualizing issues during the product planning & development phase. Due to this, testers can report issues at early stages of the product development cycle. They feel more motivated since there is a learning opportunity involved in each interaction. In some cases, the thought process of the test team can become ‘linear’ in approach.
   
   Continuous testing is performed on a more ‘polished version’ of a product since it is performed at every stage of product development & testing. Hence, the severity & type of issues that a tester comes across during the process of Continuous testing can be very different from those unearthed during Shift-Left testing. This makes the test team more informed about ‘critical issues’ and they can incorporate those learnings at the beginning of each testing cycle. The overall motivation & pro-activeness of the test team is more since there is more learning & they can utilize those learnings for the betterment of the product.
4. Continuous Integration & Continuous Deployment: Today’s modern development practices make use of CI (Continuous Integration) and Continuous Deployment (CD). CI is a common practice that is used to simplify the overall ‘software release’ process. CD, on the other hand, ensures that important features & bug-fixes are shipped to the customer on a regular & timely basis. By implementing CI, important changes are merged to the ‘production branch’ on a regular basis & the same changes are shipped to the customer using CD.
   
   Before the changes are shipped to the end customer, they have to be tested and this is where Continuous Testing can play a vital role in the verification of those changes. As the product development reaches a more ‘mature phase’, the number of CI & CD cycles would also increase. In order to make sure that a quality product is shipped to the customer, continuous test cycles of Continuous testing should also be performed (at every stage of CI & CD).

Shift Left Testing – A Subset Of A Successful Continuous Testing

So we have looked into differences between the shift left testing and continuous testing. But the question that still remains is whether we should consider them as two entirely different testing methodologies? The answer to that would be no! They offer differences, but as we looked at earlier, Continuous testing involves everything from shift-left to shift-right testing. So, it would be wrong to consider them as mutually exclusive. Rather, Shift left testing can be related as a subset of Continuous testing. When incorporated well, Shift left testing with continuous testing could unleash much faster and robust product in the market.

Conclusion

More & more organization functions are transitioning to practices like ‘Continuous Testing’ since the testing process is more severe, extensive and thorough. However, in order to achieve a better ROI, it is crucial to integrate Shift-left testing with Continuous testing. Shift-left testing should serve as the foundation for your continuous test cycles. Also, before your organization functions transition to DevOps i.e. Continuous Integration (CI) & Continuous Deployment (CD), make sure that all the ‘dependent functions’ e.g. marketing, business development, support, documentation, etc. are ready for the change since each ‘building block’ is critical in the process.

Himanshu Sheth

Himanshu Sheth is a seasoned technologist and blogger with more than 15+ years of diverse working experience. He currently works as the ‘Lead Developer Evangelist’ and ‘Senior Manager [Technical Content Marketing]’ at LambdaTest. He is very active with the startup community in Bengaluru (and down South) and loves interacting with passionate founders on his personal blog (which he has been maintaining since last 15+ years).

See author's profile