To really understand how your website performance will hold up—or not—under holiday stress, and to understand what the experience will be like for users, use an arrival rate methodology and factor in behavior models for the many, many types of users and tasks your site will serve.
Behavior modeling results in numerous permutations (often thousands) combining these variables:
- Familiarity: experienced users vs. newcomers
- Connection speed: super-fast FIOS vs. super-slow mobile device, and everything in between
- Latency tolerance: patience of users with slow site response
- Interaction speed: complexity of the page to navigate, and attention level of the user
- Tenacity: willingness of users to stick with a task through completion
Test in the real world—all of it.
To know how your site will perform for users dispersed across the country or the world, load testing must be done over the Internet, from the same geographic locations as your users, not from behind the firewall. There’s simply no way to simulate the vagaries of Internet backbones, third-party content feeds, CDN performance, and signal transmission through the critical last mile—unless you are at the end of that mile, with a browser.
With testing agents dispersed where your users are, you get an accurate picture of variations in performance, and overcome the danger of looking at averages. An average page-load time of three or four seconds may seem OK, but that kind of average could mean your page is loading in one second for someone in New York, but taking six or more seconds for someone in Chicago. And that is not likely to be acceptable. The solution is to test from multiple, geographically dispersed locations, look at the data, and address any local or regional bottlenecks.
The holiday shopping season is the culmination of many hard hours of work for the IT/Web department. And no matter how well things are planned, no matter how rigorously everything is tested, there’s always the chance that the unexpected will happen and something will go wrong. So it makes good sense to have technical personnel on hand and on call during all the critical shopping periods to handle any emergencies, and to have extra computing capacity standing by just in case it’s needed.
Read More at http://keynote.com/benchmark/new_media/article_streaming_for_primetime.shtml