The world is changing every second, so should the software delivered to the world. Very often change in a single line of code of the existing or developing software has a massive impact on the rest of the software and its features. In such cases, software development companies have to go back to these features and double-check that they work correctly after the new features are added. And this is where regression testing comes in handy.
What is regression testing?
Regression testing is a type of software testing targeted to the verification of the existing code and system functionality against the newly introduced changes in the code. Regression testing can be performed either manually or with the help of automated tests.
Regression testing examples can be divided into two major groups: full regression and iteration regression.
Below you will find an illustration of how regression tests work in a real-life software development process. Please note that this is a hypothetical case based on existing kinds of software and their probable issues.
Regression testing examples: .gif app development
This hypothetical app is designed to create .gif images with the help of a mobile device. Owing to the nature of the app, it should store, display, and process video and photo materials as well as perform a particular set of operations to create a moving .gif image. Such an app should have access and permissions to use the device’s storage places (gallery, memory card, etc.) as well as the device’s built-in camera to record materials for the moving images.
Let’s assume that the development team is working within the Scrum framework, and hence needs to deliver some milestone results every second week from the project’s start date. Agile Scrum framework suggests that every two weeks a bigger team meeting is held to reiterate the progress, discuss the issues, and plan strategies to resolve them during the upcoming iteration period. This period is called a sprint.
Manual regression testing
Whenever the end of the two-week period comes, the quality assurance engineers perform manual regression testing. By definition, it relies on the existing test cases that are defined over the two-week period by both developers and testers. Before completing the sprint, there are always sanity tests that consist of a particular number of test cases. For example, let’s take 200 tests. In order to keep the sanity tests within a reasonable time and workload frame, the testing team has to revise the test cases every sprint to keep no more than 250 of them. In this way, the outdated and no longer appropriate test cases are deleted and replaced with the new ones.
If we consider the time required for such tests, it is usually about 20% of the scrum period. Manual tests on the UI of an app are essential since they allow to identify major and hence most common app discrepancies that have to be fixed before the team continues toad the next app layers of the code.
In terms of manual testing on our example of a gif app, it included clicking through all buttons and views of the app to ensure that everything works properly. For instance, after adding the button “Share” the app crashed. This would be a major defect in the proper operational roadmap of an app. So if it is reported right after the button is added, the development team can go back instantly and fix the existing issue.
Then let’s assume that, after the “Share” button, the team adds a feature called “Share on multiple accounts” within the existing sharing feature. In this case, the QA team gets back to retest the new feature of the existing button and hence conducts regression testing. Very often, during such sprint reviews and regression tests, it appears that a new feature blocks an existing one so the code needs to be revised and fixed.
Manual testing can detect some major issues, yet test automation so far remains a more progressive and detailed regression method for long-term projects.
Automated regression testing
It is obvious that even the best and biggest test team is unable to verify every new and old feature of any app after every two-week update. So while manual regression testing is great, still automation is the key to success. During automated testing, it is possible to identify defects that appear randomly in the app so they are almost impossible to be found during manual testing. As a rule, the number of automated tests in a sprint should be held under 300. The software the QA team would use can vary depending on the project needs and requirements, however, it is highly advised to stick to the native testing tools (such as XCUITest for Apple devices) since they are reported to suit the needs best.
So let’s get back to the “Share” button example. Suppose the new feature of “Share on multiple accounts” seems to work randomly: once it appears and the next three it doesn’t. Assuming that the manual testing happened during the cycle when this button occurs, this defect would not be detected and might go into production without being fixed. With the help of automated regression tests, such a discrepancy can be easily identified and fixed within the same sprint.
On a final note
The conclusion is simple: regression testing is an indispensable part of any software development project. It can save a lot of time on fixing the defects at earlier development stages hence cutting the budget needs for a project.
In terms of the choice between manual and automated regression tests, both should be present in a project. However, if the team does not have the required resources to conduct regular manual checks, then it is essential (and one may say even obligatory) to leave automated regression tests that are frequently reiterated and thinned out to eliminate the old and irrelevant test cases. Whenever the budget and resources allow for it, manual regression tests should also be conducted regularly.