Strategic insights
Habits and Expectations can Render Usability Tests Invalid

Habit and expectations are two very important elements in good products. If you make a product, it will be perceived as good if it supports the habits of those using it. If it also works as people expect it too you are almost guaranteed that it will be successful.

There is just one catch - people's habits and expectations are rarely efficient in new products. Especially in products that is introduced to increase it.

A good example is "instant save", a technique I am introducing in many of my latest web-applications. In a normal web-application, when you want to edit something, you make your changes and choose "save". Instant save removes this extra step - instead it saves automatically as you edit the information.

From an efficiency point of view this is a good way to remove unnecessary steps. But, then you test it - and find that instead getting a faster system, you are getting confused people and a rather slow rate of editing. Your usability test clearly shows that your new system is user-hostile.

What is wrong? How can less work mean slower performance? Habits and expectations is what are wrong. People expect that in order edit something on a web page, they need to "save (send)" the information before anything happens. When they cannot find the save button they think something is wrong - or that they might have overlooked something.

Dilemma

In theory, and most likely with training, your new product would have increased efficiency. When doing usability test it is inefficient - even more inefficient than the previous product. You are suddenly standing in this awful situation where it seems that you have failed.

This is one of the situations where you must rely on your theoretical knowledge - and simply discard the usability test. The test is still important, because it show where people are having problems. You can use it to focus your efforts on the right places in order to change people's habits and expectations.

This also highlights the importance of prototype testing. When you test issues before they are made, you can prepare people for the change. And, if you can change people's habits and expectations before they try it, you can ensure a highly efficient and usable product from day 1.