Managing Rich Internet Applications [5]

This is the fifth post in a series devoted to the challenges of Service Level Management (SLM) for Rich Internet Applications (RIAs). In these applications, some processing is transferred to the Web client while some remains on the application server. Previous posts introduced the subject and the SLM topics I plan to address, reviewed RIA technologies, and introduced The RIA Behavior Model.

Measuring RIA Responsiveness: Introduction

Having been sidetracked by my interesting but futile search for the origin of the saying 'you can't manage what you can't measure', I am now almost ready to examine some of the challenges of measuring Rich Internet Applications. But first I must spell out the motivations for measuring an application, and explain why the choice of an active or passive measurement methodology is of only secondary importance to this discussion.

Why Measure Applications?
Reasons for measuring application performance fall into two classes, one client-centric, the other server-centric. They are:

1. MEASUREMENT: Determining how quickly the users can achieve their goals:
   
   
   • (a) Verifying that the application meets its service level objectives
     
   • (b) Detecting abnormal application behavior (typically, slow response)
     
   • (c) Identifying bottlenecks in application components
     
   • (d) Comparing builds, versions, or releases of an application
     
   • (e) Comparing two applications (e.g. our app vs. competitor's version)
   
   
2. LOAD TESTING: Determining how a system behaves under increasing load:
   
   
   
   • (a) Verifying that the system will be able to handle the projected traffic
     
   • (b) Determining how many users a given server environment can handle
     
   • (c) Predicting bottleneck components as workload levels grow
     
   • (d) Comparing builds, versions, or releases of an application
     
   • (e) Identifying components that fail after extended use

As I have indicated, activities in the first class are conventionally called 'measurement', while those in the second are referred to as 'load testing', or simply 'testing'. So although both classes require us to address many common measurement problems, I will simplify my description of these problems by using the conventional terminology to distinguish the two classes of activties.

Active vs. Passive Measurement
A few years ago I edited a version of the Network Measurement FAQ originally produced by a CAIDA working group, for publication by CMG. The online FAQ seems to have gone into hiding, but you can still download the CMG paper, Fundamentals of Internet Measurement: A Tutorial. Although it is about network measurement, a couple of paragraphs introduce the concepts of active and passive measurements:

Passive measurements are carried out by observing normal network traffic, so they do not perturb the network. They are commonly used to measure traffic flows, i.e. counting the number of packets and bytes travelling through routers or links between specified sources and destinations.

Active measurements, on the other hand, are performed by sending test traffic into the network. For example, one might measure a network's maximum carrying capacity by sending packets through it and increasing the sending rate until the network is saturated. Clearly one needs to be aware that active measurements impose extra traffic onto a network and can distort its behavior in the process, thereby affecting measurement results.

When measuring Web applications, similar definitions and concerns apply, except at the level of application traffic and the server infrastuctures that deliver applications. While load testing requires an active measurement approach, application responsiveness can be measured using either active or passive methods.

However, no matter which is used, the passive and active measurement approaches differ only in the way application traffic is generated -- both still require mechanisms to measure that traffic. Passive measurements must capture the behavior and experience of real application users, while active measurements must do the same for synthetic users, that is, computers mimicing the behavior of real users.

This is fortunate, because to discuss the pros and cons of real and synthetic monitoring would require a major blog post. In fact I gave a short presentation on that topic at Interop last year, which would provide the structure required. But for now I will simply note that both real and synthetic application users have to be measured, and RIAs present many common measurement challenges, which I will be describing in my next post.