The most challenging presentation I saw at Stareast was by Google
Senior Test Engineer, Goranka Bjedov. She makes the case that the world
is heading toward developing software without testing for quality and
that this practice may not be a bad thing. Scary but true!
First,
Goranka pigeon-holed testing into two categories; productivity and
quality. Her definitions (per my notes) are as follows:
Productivity
Testing – Making sure programmers don’t break code (e.g., unit
tests). Testing things consumed by machines. Anything consumed by
machines is easy to automate. These tests are cheap, fast,
well-defined. The problems failed tests expose do not require deep
analysis.
Quality Testing – Testing things consumed by
humans. Anything consumed by humans is not easy to automate and
therefore difficult to test. Expensive. Tests become more flaky as the
system becomes more complex. The right tests are not clear. Failed
tests require deep analysis. These tests take longer.
With the
promise of quicker software delivery, productivity testing has become
more important than quality testing. Wake up, the world is already
adapting in several ways.
For example, at Google, they know
hardware and infrastructure will always fail. Instead of wasting time
with exhaustive tests, their solution is to manage risk (e.g., build in
seamless failovers and backups) and shield the user from the failures.
Goranka
also countered that in cases where poor quality is seemingly not
optional (e.g., medical software) users have already adapted by not
relying on it. She claims users in hospitals, for example, know not to
trust someone’s life in a piece of software. Instead, they monitor the
patient as a human and understand that software is fallible.
These
are excellent points, IMO, and I would have been satisfied
contemplating a future where my job no longer existed...but hold on!
Goranka
asked us to do a little exercise. She asked us to determine the rule
used to generate these three sequences by writing five additional
sequences of our own:
-25, -5, 15, 35, …
2, 4, 6, 8,
…
0, 3, 6, 9, …
I don't want to give away her rule,
but you can still try it on your own.
After surveying the
audience, she pointed out that developers tend to write confirmatory
tests more than testers, who tend to write more negative tests. Thus,
perhaps testers do play an important role. She also questioned how much
productivity tests actually tell us about the system as a whole. Her
answer? …they tell us nothing.
In the end, she left us with
this thought…
If you think (non-programmer) testers are
important, you better start doing something about it.
