TDG Blog

Does Cognitive Bias Make Software Testers Miss Out Bugs?

Software testing has witnessed a revolution in the past decade or so, with the introduction of test automation, through advanced test automation tools. Test automation today is powered by advanced technology that opens doors to new possibilities, like executing multiple test cases simultaneously, testing across multiple browsers and platforms, integrations with other popular tools and recording test scripts for reuse. What’s more, QA teams today can use these tools to fetch detailed reports for the analysis of test case executions and understanding at which step the test has failed, as well as the possible root causes. Yet, in spite of having such advanced technology within our reach, we still see testers ask themselves the question ‘What made me miss this bug?’

In other words, it’s a common occurrence for testers to miss bugs that go unnoticed and undetected during the initial phase of testing, only to be discovered later on when the product is deployed to production, or worse still, after the product is launched in the market. This makes one wonder –

Why do Testers Miss Bugs?

There might be several reasons for this and the reasons are likely to vary from one QA professional to another. However, considering the fact that today we have the best technology we could ever want in the new age test automation tools , experts believe the reason lies in the way testers approach testing, which in turn is defined by the biases that govern their minds. Experts say we are all governed by a set of biases that are a result of our past experiences and they use the term cognitive bias to define this. They say a missed bug is nothing but an error in judgement which is an outcome of cognitive bias.

What is Cognitive Bias?


Source: https://bit.ly/2WsS248

Wikipedia defines cognitive bias as a “systematic pattern of deviation from norm or rationality in judgment. Individuals create their own “subjective social reality” from their perception of the input”. What this implies is that cognitive bias makes individuals to think irrationally or illogically which results in an error in judgement.

The human brain and how it works never fails to surprise us and the way biases influence our thought process is just another intriguing aspect. Talking about how cognitive bias influences testers, testers are governed by their biases when it comes to where to look for bugs, which functionalities would require a more intensive testing approach, how the user is likely to interact with the software or application, an assumption about the app based on who developed it, and the list goes on… What makes it all the more interesting is that there are not one or two but an entire set of different types of cognitive biases!

Different Types of Cognitive Bias

So, let’s learn a little about the ones that are the most commonly found and understand how they are likely to affect the thought process of QA professionals.

1] The Confirmation Bias
 
Source: https://bit.ly/2U2T7D1

The confirmation bias is the term used to define the tendency of individuals to “look for” or pay attention to information that confirms their long held beliefs. This bias makes individuals stick to their own perspective, and prevents them from analyzing the other perspectives and arriving at a logical and rational conclusion.
In the testing world, this bias would make testers spend less time testing the code by a developer who he thinks writes impeccable codes!

2] In-attentional Blindness
 
Source: https://bit.ly/2uzJP2i

It is a term that implies missing out the details, even though they are the most obvious, just because one is not looking for them. In other words, it is referred to the phenomena of missing out on details that one would not have missed out if there wasn’t a lack of attention.

When it comes to QA, this could mean focusing all attention on testing a new update or a newly added feature and missing out on defects in other functionalities or aspects of the software.

3] The Negativity Bias and the Optimism Bias
 
Source: https://bit.ly/2FI8Mih

The negativity bias and the optimism bias are opposite of one another and which one applies to an individual is likely to be influenced by his/her general outlook towards life. While the negativity bias makes an individual expect a negative outcome for every endeavor, the optimism bias makes him/her too optimistic.

In the world of software testing, negative bias may make a tester have low motivational levels to go the extra mile to test for bugs as they believe it’s not possible for them to create a bug free software anyway. The optimism bias, on the other hand, may make the tester feel so optimistic when they find no bugs within the first few hours of testing, that they start believing the software is virtually free of any defects whatsoever, ignoring the possibility of defects in other areas of the software.

4] The Congruence Bias

 
Source: https://bit.ly/2FsURes

The congruence bias makes an individual to look for possibilities defined by his/her own hypothesis, without considering the possibilities as defined by any other hypotheses. In other words, it means creating a positive outlook about something if it meets all the criteria one has in his/her head, without even considering the fact that those criteria might not be enough to form an informed opinion.

In the testing world, the congruence bias makes testers test a software only for the expected behavior, thus completely missing out on the aspect of negative testing. This way, they can never be sure if the software is behaving in ways it SHOULD NOT.

5] The Bandwagon Effect

 
Source: https://bit.ly/2YxMkja

A very commonly observed phenomenon, the bandwagon effect is a term that is used to describe how we as individuals tend to get influenced by the opinions of others. This is what makes people buy products that are advertised by the influencers who are popular and are idolized by us.

When it comes to testing, the bandwagon effect comes into play when a tester is influenced by the opinions of his peers or team members and may not spend much time and effort in testing a software if the team members feel it is relatively bug free.

6] The Representative Bias

 
Source: https://bit.ly/2TZYPFy

Also referred to as the resemblance bias, the representative bias refers to looking at a situation or scenario with an outlook that is influenced by one’s experience in similar situations. In other words, individuals affected by the representative bias use their past experience with similar situations as a representative for the actions to be taken in the current situation.

In the testing world, QA professionals may only look for a standard set of errors in a web application testing project, instead of treating each web application as an individual software that may have its unique set of defects and issues.

7] The Anchoring Bias

 
Source: https://bit.ly/2gGXJfo

The anchoring bias refers to an individual getting influenced by the very first piece of information about a product, a person or a place. This first piece of information that they come across can be a numerical value, a certain quality or the first review – it can be anything. The thinking process of the individual is then anchored to this bit of information and is also influenced by it.

When it comes to QA professionals, the anchoring bias can make testers refer to the set of product specifications blindly while testing the code, instead of ensuring the specifications match what the user behavior and user requirements.

The Final Word

So, you see that there are quite a few different types of biases that can affect testers and hinder their ability to find the maximum number of defects and bugs. To minimize the influence of the biases on our rational thinking, it is essential that we train our mind to look beyond what the biases make us to believe. When a tester identifies that his/her rational thinking is being affected by a particular bias, it becomes easier to overcome it.

Have you ever experienced any of these biases? If yes, how did you overcome it? Share your experiences with our readers in the comment box below.
Cognitive Bias, QA, software testing, test efficiency, testers
Write a comment
Cancel Reply
    Search
    Subscribe Us
    Relevant Posts
    Author's Recent Posts
    Categories