Skip to content

Part 2: Scrum From a QA Perspective

Analyzing Scrum

Part 2:

There is an excellent blog that describes software development as engineering a bridge:

Imagine joining an engineering team. You’re excited and full of ideas, probably just out of school and a world of clean, beautiful designs, awe-inspiring in their aesthetic unity of purpose, economy, and strength. You start by meeting Mary, project leader for a bridge in a major metropolitan area. Mary introduces you to Fred, after you get through the fifteen security checks installed by Dave because Dave had his sweater stolen off his desk once and Never Again. Fred only works with wood, so you ask why he’s involved because this bridge is supposed to allow rush-hour traffic full of cars full of mortal humans to cross a 200-foot drop over rapids. Don’t worry, says Mary, Fred’s going to handle the walkways. What walkways? Well Fred made a good case for walkways, and they’re going to add to the bridge’s appeal. Of course, they’ll have to be built without railings, because there’s a strict no railings rule enforced by Phil, who’s not an engineer. Nobody’s sure what Phil does, but it’s definitely full of synergy and has to do with upper management, whom none of the engineers want to deal with, so they just let Phil do what he wants. … After the introductions are made, you are invited to come up with some new ideas, but you don’t have any because you’re a propulsion engineer and don’t know anything about bridges.

The entire article is fantastic, and I highly recommend you give it a read. This introduction, though, perfectly summarizes the main issue I’ve experienced in software, and one that scrum implementations often fail to address. The scrum team determines how to provide the requested solution, but the people deciding what they need can all have conflicting goals, understandings, and expectations. It’s an important distinction: in theory a business should provide the details of what they want, e.g. "I want software that solves my issue with billing customers for my service, so it is easier for them to pay, and so that I do not have to keep a physical book of payments," and the scrum team determines how, e.g. "We’ll leverage an off-the-shelf solution to handle the incoming payments and use some custom logic to ensure this is represented correctly in an electronic system of record" (to put it very high level. An actual plan would be more detailed and span several user stories.) In my experience, when the "what" is not a consistent, well-thought-out message, no amount of good implementation will produce a good product. Unfortunately, given agile-like methodologies are often touted as ways to "fix" a "broken" development model, there is an expectation that vague requests can be turned into high quality software quickly. Or, more accurately, that vague requests can be turned into what the user wanted, and it’s high quality. Instead, work requests go from a vague "I want software that solves my issue with billing customers for my service," to, "I had a customer complain about how I accept payments, I heard PayPal can handle this, use PayPal to get payments from my customers" with no explanation on what the complaint was, the various payment scenarios that need to be handled, and how sophisticated any bill tracking software needs to be.

This is the functionally equivalent but opposite issue most companies run into with waterfall. Continually interrupting sprints, or having vague requirements that are expected to "fail fast" and cause rework is no better for the development process than dogmatic adherence to a set of requirements. In either case, engineers are not empowered enough to help influence important decisions:

Waterfall replicates the social model of a dysfunctional organization with a defined hierarchy. The most interesting work is done first and declared complete, and the grungy details are passed on to the level below. It’s called “Waterfall” because communication goes only one way. If the design is bad, it must be implemented anyway. (The original designers have probably moved to another project.) Agile, then, replicates the social model of a dysfunctional organization without a well-defined hierarchy. It has engineers still quite clearly below everyone else: the “product owners” and “scrum masters” outrank “team members”, who are the lowest of the low.

There’s a follow-up to that blog that delves further into the management issues, but the point is clear: the software teams are not empowered to affect change at a meaningful level, or have adequate representation at a high enough level.

Missed ship dates. Incomplete feature sets. Low-quality software (of particular importance to QA.) All are symptoms of teams with no empowerment. These are the pitfalls of the expectation that comes with agile teams. With no senior engineer or architect to tell a client or manager that their request is unfeasible, vague requests are made for things like "swap out one system for another; it should be a straight 1 to 1 replacement. Have it done in a month." When it takes a week of asking these same people "What if there’s a bug in the current system? Should that be fixed to work with the new system?" and two weeks of implementation with no one thinking of QA, the team is now down to a single week of testing a release candidate.

The problem is two-fold. When the expectation from non-technical managers is that "agile means they can change on a whim, and deal with my vague requests getting more specific as we get closer to the deadline," the rapidly approaching deadline means developers have to work longer (which does not equal better) and QA has to put even more time to not only account for the late changes, but also for the inevitable host of additional bugs that crop up which must be investigated. Forget about regression testing.

Scrum is supposed to empower developers to decide the best way to fulfill a clients’ wants. But it still prevents them from giving meaningful input into the "what". As any freelance graphic designer can tell you, clients often do not know what they want, just that they have a problem (and I’ve done freelance 3D design work, and that video is absurdly accurate to how meetings actually go down). And yes, tactfully telling a client what they want is often a good way to go. And yet, clients often try to dictate the "what" (and sometimes the "how", as is often the abuse of waterfall) but do not have the experience to effectively do this on their own. They change their requirements without changing due dates, and the developers have no power to change the due date themselves. They assign additional projects before existing projects are finished. Or they request a feature that is not within the scope of the team’s specialty, but may be within another team’s specialty at the same organization. They do not organize projects that span teams, so collaboration is difficult and a hindrance. The cost of doing this is significant. It results in churn, burnout, missed dates, and a product the customer still isn’t satisfied with that the developers weren’t happy delivering, but were overridden every step of the way "because the team is agile, so they can change what they’re delivering."

Remember the article above? About how NASA has near-perfect software, and they achieve this by a strenuous rigor surrounding their design, implementation, and test? Most teams can’t achieve that. Most teams shouldn’t try to achieve that. The amount of money and time it would take for the average software provider without life-or-death consequences of their software to do that are unnecessary. But there is an important point: when the article talks about how their software is designed, it is designed with technical people in the meetings. There is an explicit agreement between NASA and Lockheed-Martin on what the software should do, and the developers code to that spec, and the testers test against that spec. Planning before implementation is not antithetical to scrum (as I often hear in rebuttal to this idea.) Basically, many businesses still treat a scrum team how they would treat developers in a waterfall environment, with meaningful communication and high level project decisions still being made at the top and communicated down, now with the added downside of the expectation that the team can change quickly and often. Instead, software development should be a joint effort of agreeing to what the delivered software should do, how long it will take, what it will take to ensure it meets or exceeds quality standards, and what sort of support occurs post-release.