Beware of false consensus and pluralistic ignorance
False consensus is a cognitive bias where you exaggerate your ability to guess what other people think. What’s more, when you find you have predicted other people wrongly, instead of saying to yourself, “Oh, I got that wrong” and modifying your view, you dismiss these people as ignorant or exceptions and persist with the same view you had before.
Pluralistic ignorance is when people conform to an opinion they privately don’t agree with, because they think they are odd or in the minority, when in fact most people think like them.
An example from user testing
Here is a common scenario you come across in user testing. Let’s say you are testing a form with a blue button in it. Here is a dialogue with two users.
User 1
User: “Oh, that’s a horrible blue on that button. No-one will like that blue.”
You: “I noticed you took a second or two to find the button.”
User: “Well, why did you put it over there? People won’t see it there. It would be much better over here. Everyone will see it if you put it there.”
User 2
User: “I must warn you, I don’t know anything about websites. It’s no use asking me about websites.”
You: “It’s fine. You don’t need to. You are testing the website, not the other way round!…I noticed you clicked on that button. What do you think of the colour?”
User: “Oh, I think that’s a lovely blue, and it makes it stand out.”
You: “You didn’t take long to find the button. Would you have found it even quicker if it was anywhere else?”
User: “No, I thought it was obvious where it was, but that’s probably just me!”
User 1 displayed false consensus. They have an exaggerated idea of their ability to second-guess other people. User 2 shows pluralistic ignorance. They are self-effacing, think they are in the minority and are hesitant to express their real views.
When you are assessing your own site you have to bear in mind these two phenomena. Often a more opinionated person (usually at a senior level) will declare that everything is fine and that any negative feedback can be discarded because it came from people who aren’t target users, or who are exceptions.
This person may be in a meeting with less self-confident or less senior people who go along with them because they think everyone else agrees with them, and that they themselves probably have an eccentric, uninformed opinion.
The way out of this is to assume you don’t know anything about other people. Every guess you have is just a working hypothesis to be tested. Other people (users and staff) are a hugely valuable resource for feedback and ideas. You need to create an environment where everyone is encouraged to give feedback and the feedback is treated seriously.
Case study: resistance to evidence
A friend worked on the website of a major national charity in the UK. The web developers went through a redesign with an external agency. After several months they come up with a new swish design with an eye-catching banner photograph on every page.
Just before launch, they organised a user testing session and found that people were getting lost and confused on the site. It turned out they didn’t notice the main navigation because it was a thin bar above the banner photograph. Their eyes skimmed right past it, and they looked for main links in the page body.
What was interesting was the response of the web team and agency: they dismissed the testers and not representative, and kept the design as it was.
In the CAF bank website relaunch (see Case Study in Test little and often), many users felt frustrated that the official statements appeared to be blaming the users rather than the system. Even though the new system lacked key features, the statements from the bank seemed to imply users just needed to get used to it i.e. there’s nothing wrong with it.
Look for evidence that you’re wrong
The philosopher of science Karl Popper (1902-94) emphasised the importance of falsifiability in scientific theories. It is relatively easy to gather evidence that supports a theory, he said, but the real test of a theory is if there is evidence against it. A mass of evidence for a theory does not prove it, but a single piece of counter-evidence could disprove it.
The example that Popper gives is of white swans. We could keep going to the same lakes and rivers and see only white swans, and think “all swans are white”. But if we actively followed up reports of black swans then we might discover a black swan, and our theory is disproved.
There is much more to Popper’s theory, of course, but the lesson in web development is to keep looking for evidence that you’re wrong, and that your site isn’t working as well as you think. You need to keep checking by listening to feedback and watching people use the site. It’s criticism that will help you improve the site, not defensiveness or self-delusion.