Why It’s Crucial to Catch Your Broken Functionality Before Sending it to Production
QA resources waste time through long, protracted test efforts that try to hammer all of their bug testing through the UI at the end of (or after) development. One of the main problems with this approach is that it becomes tough to isolate functionality so that we are only testing one thing at a time with a known codebase. Isolating variables can help produce more reliable, accurate results in testing.
Let’s say there’s a defect at the tail end of a long development cycle; we’re three months into working on a new feature, and I find out I can’t log in with one specific password. It’s a valid password, so I file a defect. By the time the developer looks into it, they end up several layers deep trying to modify a class potentially written by someone no longer at the company. All the developer knows is that when they tweak the class to accept the broken password found by QA, it breaks their own test for validating passwords.
If we isolate variables appropriately and quickly, then when that one thing breaks we can write a concise, accurate bug report for the developer to turn around and implement a fix, ideally within a sprint of the original implementation. If we test one thing at a time, we have a better idea of what is broken.
The Ultimate Goal: No Broken Functionality Sent to Production
If functionality is broken, then it can be fixed quickly and efficiently before production. To that end, proper functionality isolation happens at various levels of an application and has many benefits. Here’s a brief list of isolation points and some example associated testing methodologies:
- Individual functions (automated unit testing)
- Data access layer (automated service tests)
- Data repository (boundary analysis, referential integrity checks)
- Integration layer (automated service tests, integrated UI tests)
- UI layer (functional UI tests)
Find some excellent concrete examples in this overview of the Practical Test Pyramid.
Which Bug Testing Category Does Your Team Fall Into?
One benefit to isolating these various systems is that, should a defect occur, we can provide better results (whether it be a bug report or something else) because we can identify the layers affected by a defect. If we think of bug testing and bug reports in a progression of least informational to most, teams tend to fall into one of the following categories:
-
No testing/No bug reports: Either the team doesn’t test, or there is a critical disconnect between developers and defect reports.
-
Unfocused testing/Inaccurate bug reports: In this case, bug reports aren’t indicative of the actual problem. These tests and bug reports are prone to producing red herrings and require the developer to spend inordinate amounts of time investigating issues that end up either not being issues or take more time to reproduce than they do to fix.
-
Good testing/Good bug reports: Tests are focused on fulfilling a need (testing for a business requirement, testing for a functional requirement, etc.) and bug reports have accurate reproduction steps for how the issue occurred and approximately when the problem started.
-
Great testing: Tests effectively isolate different functionality in the application, and as a result, bug reports can isolate the exact layer that failed, when the failure started, and list accurate reproduction steps.
Strive for Good Testing or Great Testing
Good testing offers some known room for improvement, and it helps the developer quickly identify what is wrong. It could be the highest attainable goal if a QA person is not working directly with the dev team.
By having QA and devs work together to identify and test small pieces of functionality quickly, the turnaround on bug fixing will often be quick and painless. Prolonging the testing cycle, moving it to a group with little contact with the developers, or bug testing without this mindset will produce bug reports that are hard to identify and fix.
I am a big fan of teams that continually assess their current strengths and weaknesses and try to increase their efficiency, quality and reliability by maximizing their strengths and improving on their weaknesses.
Where Would You Rate Your Own Team on the Effectiveness of Their Testing?
While thinking about it, don’t forget that you are not alone in trying to figure out how to get your team to the next level.