Delight, Satisfy, or Frustrate?

My recent posts have discussed the relationships among user expectations, site responsiveness, and user satisfaction. To sum up the conclusions of all the research I have seen (and done) into Web download times, Web users’ tolerance for delays depends on several factors, including their expectations, site feedback, the complexity of a task, its importance, and the relevance (utility) of the information being provided by the site. But their perception of a site’s quality and credibility diminishes as its download times increase.

Today I am going to shift my attention to performance management. For an eCommerce company embarking on a program of Service Level Management (SLM), what are the implications of all this research? How should you set response time objectives? Here are my conclusions:

• Treat Miller’s thresholds as a design framework to be kept in mind (more on this below).
  
• Forget about Bickford's 8-second rule! Web norms have progressed since 1997.
  
• What really matters are the service levels your customers expect. You can delight them, satisfy them, or frustrate them by delivering levels of responsiveness that they perceive to be fast, reasonable, or slow respectively.

Writing about the role of evolution in site design, Jared Spool observed:

"For most types of sites, there are existing models already out there. Someone who wants to produce a new site with pharmaceutical information, for example, can find dozens of existing pharmaceutical sites. A small investment in studying how users interact with existing sites can reveal a lot about what works for your users on their tasks. You could easily develop an understanding of the best practices and, from that, produce your own guidelines".

The same applies to your users' perceptions of performance, and how those should determine the objectives you set for your site. Because those perceptions will have been set by other Web sites (including your competition), the performance of comparable online services delivered by other sites in your market segment should be your starting point. You can never go wrong by measuring your competitors and striving to match them.

For example, Keynote's Public Services division publishes weekly indexes of Web site performance for several industry verticals. Many companies in these markets monitor these indexes as an important benchmark of the competitive climate, and of their own site’s quality.

Some leading companies have taken this approach several steps further, setting up comprehensive measurement, tracking, and reporting programs for their most important Web initiatives, and letting their measurements of the competition drive their own service level objectives. One company even recalibrated the annual performance objectives and bonuses of IT and development staff, based on their ability to match competitors’ performance. And it works! The company improved from last to first place in its industry by following this SLM strategy consistently over a 3-year period.

So if matching the competition is everything, how are Miller’s thresholds relevant? I see them as absolute standards within which you will frame your relative (competitive) objectives. Although you should aim to compete, you can relax a bit more if you are already close to meeting Miller's guidelines. For example, if your competitors can find widgets in 4 seconds, your site's 8-seconds response is too slow. But if they take one second and your site takes two, you are doing OK -- focus your tuning efforts on another online application that needs help. For additional perspective on this subject, I recommend reading this 2002 exchange in Business Communications Review between Peter Sevcik and Peter Christy.

A related aspect is the nature of the service your site is providing, in response to the user's input. Miller proposed that ideally all computer responses should occur within two seconds. But in practice, it is reasonable to absolve some complex interactions from this 2 second target, if you are sure that the customer would not expect the response in 2 seconds. Credit card validation is a good example. If in doubt, be guided by what other sites can achieve for a comparable function. The subject of Web "norms" for various types of interaction is one that I plan to revisit here. It is already being discussed in the context of the Apdex approach to setting objectives and reporting response times.

To sum up, performance matters! It affects your bottom line, if you are doing business online. And to win at eCommerce, you cannot be content just to match your competition at every turn. Look at the situation from their point of view. Their customers’ expectations will also be based on the prevailing Web climate. So if your site is faster, the competition can no longer delight their users with their responsiveness – the best they can do is satisfy them. This gives your site an edge in the usability stakes, which a competitor must now try to compensate for by doing better in some other aspect of their site. If they don't, and remain noticeably slower for an extended period, eventually some of their customers will become frustrated, and switch to your service.

So having a faster site gives you a competitive advantage. This is where the money you have invested in your SLM program really pays off. I've been working in this area recently, and in future posts I will be writing more about the value of SLM, and how to calculate return on investment (ROI).