OK, I'm pretty much sold on the idea behind eValid...that truly real tests have to be done with truly real means, and in that case then if I have a web application then I have to use a browser.

Two questions though are still troubling me...

(1) What happens when it's an AJAX application that eats a lot of CPU time in the client side -- after all, this is why we do AJAX, to minimize load on the server in favor of doing it in the client?

(2) What about I/O limitations...don't you run out of I/O capacity when you have 125 browsers running at one time?

-SK

Thanks for writing. Those are good questions.

(1) Basically, what you're asking is, does an AJAX application that uses
JavaScript code locally eat so much computer time that it would affect the
reality of the loading experiment?

The answer is, maybe yes and maybe no: it depends on the application.

In most cases JavaScript in an application most affects performance in the first
page or two that is loaded into the browser space. That is when the majority of
the JavaScript code is transferred in.

After it's loaded in, from what we can tell at least, most AJAX applications
don't consume a lot of CPU resources.

Not zero or 1% CPU utilization certainly, but not pegged to 100% either.

There are some notable exceptions and usually these are related to the more
sophisticated email applications, in which almost all of the display generation
and message processing is done locally. So in those cases we do see 100% CPU
for a time.

(2) Input/Output traffic can also be an issue, and to be honest the combination
of heavy I/O traffic PLUS heavy CPU load can cause some skewing of loading
results.

Gmail, to take a typical example, downloads about 2.2 MBytes just to open up an
account, includes about 75 URLs, and takes sometimes 15-20 seconds to get ready.

So for server loading you have to be aware of which tests impose high I/O
traffic, and also which ones have a big download footprint.

You can get the data on this easily from eValid's EventLog by simply running the
test as a singleton. The Playback Summary Record shown at the end off the
EventLog gives you all of the key facts about that particular playback.

So, in direct answer to boty (1) and (2), there are functional tests that would
run you out of CPU and I/O capacity in a 125 parallel browser situation. With
careful planning -- and knowledge of the details in advance -- your loading
experiments will be perfectly valid.