“I’ve tested everything, there are no bugs”
Why “Safety Language” is so important in Software Testing
As testers, before getting ready for release we may get asked
“Have you tested everything? Are there any bugs?”
Or questions along these lines. Rightly but perhaps more wrongly, people look to us as testers for reassurances, and sometimes (again wrongly) to “sign off on a release”. I must admit that when I first started out in testing this was more common than it is now. Back then I’d take their questions at face value and reply (quite naively looking back now)
“Yes, I’ve tested everything. There are no bugs or issues”.
Over the years, I’ve realised how I was answering these questions in the wrong way. It took one specific moment, to make me understand this.
I was a relative newcomer to testing at a different company with about 2 years experience in the role. I was working in an Agile team, and I was the sole tester. We were releasing quite a big change. I was asked the same question posed above. I answered it, in much the same way I noted above. Lo and behold, there were issues after the release. The product owner came to me and said “You said you tested everything”.
Of course, you and I know, I didn’t mean everything. Software Testers know that you can’t test everything. The release had an unintended effect on another part of the system. A part of the system that we (the team) had considered out of scope as it wasn’t touched (or so we thought). As a result of this, I knew I had made a mistake, I spoke to some peers at the time, they mentioned something about “Safety Language” and how I could have benefited from it.
Why does it matter?
When you deal with absolutes, you leave yourself no room for error. If an error is subsequently found, you will lose some credibility.
Credibility is important. More so as a software tester, as you need people to listen to you when you have any concerns or issues. For example, you need people to listen to you when you state what testing you have performed so that they are aware of the scope of the testing.
By using Safety Language:
- You still maintain credibility, even when there are issues.
- You’re still being open and honest about what you have tested
- You’re being truthful in that you have found no bugs. You’re not saying there are no bugs, because software can never ever be bug free.
Wait… Software can never ever be bug free?
You can never test everything. You can test the areas that you feel are at most risk. How you identify these areas as a tester is up to you.
You focus your testing on the areas that are high risk. You do this because you feel, that as a specialist in your domain, you will gain the most value in these areas. You may use automated checks to guide where you wish to test and explore.
Automated checks are like a net, in that they will stop some bugs getting through. But with any net, things will always slip through. This is where your expertise and testing mindset will come into play to focus your testing efforts.
Developing software is hard. There are many things to consider: infrastructure; code; network; external dependencies and many other things. Testing everything is impossible.
Michael Bolton puts it best:
As software testers, our job isn’t to break the software. It’s to break people’s illusions about the software.
Essentially, the software itself was already broken, we were able to highlight how it had broke and the effect on the customer and/or the business.
So what should I have said originally?
You may be thinking that there is nothing wrong with what I said at the start of this blog post.
If we look back at what was originally asked…
“Have you tested everything? Are there any bugs?”
I had performed some testing for the release, I had found zero bugs. This isn’t however what I had said. What I should have said is this:
“Yes, I’ve performed the testing as we had discussed and is documented here. I found no bugs or issues in the testing I performed.”
What’s the difference?
I’m highlighting exactly what I have tested, how I have tested and even the documentation behind it (whether that be notes, or back in the day test cases). It’s clear to whoever I am speaking to what testing has been performed and that (generally speaking) they would have been involved in the planning in some form. I’ve also highlighted that I haven’t found any bugs in the testing I have performed, which isn’t to say there aren’t any bugs in this piece of software.
What are some other examples of Safety Language?
One thing I’ve noticed, is that we can use Safety Language in a number of ways.
Next time you come across a bug, that you are struggling to reproduce, try not to say:
“I found this bug, but I can’t reproduce it”
This implies that it’s not going to happen again, or that it maybe never even happened in the first place. Every single bug that you come across as a tester happened for a reason, it’s our job as testers to uncover those reasons. If you can’t recreate it, you just haven’t found the reason yet. So instead of saying you can’t reproduce it, try saying:
“I found this bug, but I’m not able to reproduce it at the moment”
Like most things, it’s a small change in your mindset and how you convey information. Like most things however, it can be small changes that can have the most impact.
Previous I may have found a bug, I’d do some exploring of the logs and console and I think I know what the root cause is. So I go to a developer and I say:
“I’ve found this bug, this is the cause…..”
The developer fixes the issue as we discussed, I go to test it again, and the issue is still there. The problem? There can be deep lying causes for a single issue. What I should have said is:
“I’ve found this bug, this could be one cause which results in the issue…”
By taking this approach, we know that it’s one cause, but I haven’t said it is the only cause. So we are free to explore some more and find any other causes that may be resulting in the bug at hand.
Now what?
I hope that having read this post you have come to understand the role that Safety Language can have in your communication as a Software Tester. We encourage all our testers to use safety language when communicating with stakeholders.
Testing is all about information. It’s how we convey this information that is absolutely critical. It’s why great software testers are good storytellers, you need to tell your testing story, tell people what you’ve done in a clear and concise way and bring them in to your testing story to help them better understand where they fit in to it.
Next time someone asks you a question, think carefully about how you frame your response. Next time you declare something, think carefully about how it is communicated and what it might mean for you.
There is also an education piece for the people who may be asking the question(s). They shouldn’t be going to a single person for their “sign off” (as that is what they are asking for) of a release. It needs to be a more collaborative experience, where everyone gets involved and anyone is empowered to give information about the software.
Do you have any good practical examples of Safety Language? Let meknow in the comments!
