Reporting Web Application Responsiveness

In a previous post, I discussed some complications of measuring Rich Internet Applications (RIAs). In particular, I concluded that …

… to report useful measurements of the user experience of response times, instead of relying on the definition of physical Web pages to drive the subdivision of application response times, we must break the application into what we might call logical pages. To do this, a measurement tool must recognize meaningful application milestones or markers that signal logical boundaries of interest for reporting, and thus subdivide the application so that we can identify and report response times by logical page.

Today I am going to look inside the logical page, and consider what happens when the application responds to a user action. I have previously written about the stages of this process here, and in this paper.

Two Standard Service Level Metrics
Because Web pages are constructed using many separately downloaded components, low level monitoring tools can collect a ton of data about the communications activity triggered by a user interface action. And for diagnostic and tuning purposes, it is always useful to measure each separate component of the download process. But for most service level monitoring, this low level data can usually be summarized in just two important metrics: initial response time and page download time.

From the perspective of a typical user, these metrics represent two distinct events in the page download experience. After 'clicking' on a page, the first measures the apparent pause until the site begins to respond, while the second measures the time for that response to download.

The elapsed time between these two events can be large enough to affect the way pages are designed. Because pages containing many images take a while to download over slower connections, such pages are normally laid out so that the most popular links are displayed early in the page download process, allowing a user to navigate to another part of the site without having to wait for the all the page content to complete.

But while initial response time and page download time provide a good indication of the user’s experience of a traditional Web application, they may be less useful for some RIAs. In these applications, the time to complete a page download might include additional asynchronous activity that does not affect the user’s experience of the current page, such as fetching additional content (like executable script files, application data, images, or even streams) for future use within the application.

An Additional Metric for RIAs
This suggests that, while the two existing metrics are still useful, a standard scheme for measuring and reporting RIA responsiveness probably needs to also include a metric that represents the time from user interface action until an intermediate event, one that we might call page display complete. This metric would reflect the time it takes to download everything needed to complete the display of the current page.

Even that definition is ambiguous, because it may not be obvious when some pages are 'completely displayed'. I see two issues here. First, what about parts of the page outside the display window, that won’t be visible until the user scrolls? Let’s deal with that by assuming the user has a screen of unlimited size, so that the entire page could be visible without scrolling.

Second, what if a page includes a streamed video, or a series of images that are displayed in sequence, each replacing the previous one? On reflection, I conclude that the 'page display complete' event should occur when the complete display is first visible -- in my examples, when the video begins playing, or the first image is displayed. That's because measuring to the last view of the display would make this metric a lot less useful as a measure of a user's experience of responsiveness. And nowhere between those two extremes offers any possibility of generality across applications.

Summarizing What’s Needed
So now I have defined three distinct metrics that seem to have fairly universal applicability when measuring the responsiveness of RIAs:

Initial response time
The elapsed time from a user interface action (a mouse click, or any other user action that triggers a download process) until the browser places the page in a state that permits a user to perform another action (technically, the user is unblocked). We could also say that this is the time until the new page first becomes usable.

Page display time
The elapsed time from a user interface action until the complete resulting page is first displayed, or could be first displayed if the screen were large enough.

Page download time
The elapsed time from a user interface action until the complete resulting page and all associated content elements have been downloaded. This time includes all secondary or background downloads triggered by the user's action, whether or not they are required for the current page display.

These definitions provide only a rough statement of requirements for RIA measurement -- none is technically precise. In practice, any attempt to implement any one of these requirements could present problems for some applications, browsers, or measurement tools, and I will write about some of those complications in a future post. But I think it is useful to identify these generally applicable goals for measuring and reporting on the responsiveness of Rich Internet Applications.

Tags: RIA, Rich Internet Application, measurement, service level, SLM, page download, Web application, Webapp.