Performance Testing for Retail

In the broadest sense, testing falls into two categories: ongoing evaluation and tweaking to optimize daily performance, and peak load testing to determine overall site capacity and potential breakpoints.

In either case, the only way to quantify user experience is to measure what users are actually experiencing. Unless you’re the local bike shop serving only your immediate area, the testing needs to be done across a wide geography and multiple backbones. And it needs to use an actual Web browser, and go through the same type of page view sequences and transactions as a typical user would. There’s simply no other way to get a true perspective on what users are really experiencing.

GETTING A PERSPECTIVE ON EVERYDAY PERFORMANCE

Ongoing performance testing, done on a routine basis or following upgrades, patches, or redesigns, can be as simple or complex as the site itself. Assuming the back of the house is in order — that servers are performing up to speed and all of the infrastructure is optimized — the focus of the testing is on the front end. Individual pages are tested and analyzed, and those performing below par are deconstructed to identify the structural problems or elements that are causing the slowdowns.

A single element could be dragging down the page load, or it could be less-than-ideal placement of JavaScript on the page. The point is that testing and identifying the problem pages or transactions is only half the problem. The other half is being able to pinpoint the causes and correct them. A robust performance evaluation regimen accomplishes this.

RETAILERS SHOPPING FOR STRATEGIES

A few retailers are feeling upbeat about the coming holiday season. Best Buy is planning to do more holiday hiring than last year, partly because of strengthening demand for flat-screen TVs, smartphones, and netbook computers, and partly because of market share they captured following last year’s demise of Circuit City. Target plans to have 26 new stores open in time for the holidays, including five “Super Target” stores with full grocery departments; this, and more aggressive pricing, are moving Target closer to a head-to-head brawl with Wal-Mart.

Most retailers, however, are simply looking for ways to maximize their share of holiday revenue. Deep and earlier discounting is expected to be widespread. And in a throwback to an earlier era, and at least partly in response to a growing paucity of consumer credit, the layaway plan is making a comeback. K-Mart and its parent company, Sears, are both offering not only in-store layaway plans, but also online versions as well. Consumers can reserve their merchandise and make payments online, and when they are paid in full, pick up their merchandise in the store. Whatever it takes, even down to giving away free gift cards, retailers are doing to get consumers to buy, either in the store or online.

Source: http://keynote.com/benchmark/online_retail/christmas_article.shtml

The impact of web load testing on performance

What drives the financial impact of a site outage or performance issues is abandonment. It’s important to understand that visitors experiencing a Website with issues don’t result in lost revenue, per se. Only when those visitors don’t return, and/or go somewhere else is revenue lost. A shopper’s tolerance for errors is called “tenacity” in Web load testing parlance. Low tenacity shoppers bail from slow searches and hanging shopping carts in a dash.

If a load test had been run that adequately modeled the impact of the planned launch, the damage would have been done outside business hours and Target management would have been able to decide whether to make changes to their systems and retest, postpone or restructure the launch or just "risk it" and see what happens.  We don't really think they ever had the chance to make those decisions.  Surely they didn't conduct a load test that predicted such an epic fail.  But should they have?

Realistic Web load tests model site usage and shopper behavior. Systems are deployed to simulate high levels of demand from multiple geographically disperse areas. Once the load is generated, the infrastructure and application’s response are watched carefully to identify bottlenecks and breakage points as the entire mesh of the Website’s interconnecting parts are stressed. Only this level of testing can accurately inform e-commerce teams of their preparation adequacy.

Read More

Web Performance Varies by Geography and Network

Unlike Web content downloads to desktops, mobile downloads can vary dramatically based on time of day, network operator used, and geographic location. For example, a mobile Website can take twice as long to download in San Francisco as compared to another operator in New York, London, or Tokyo. If a content-monitoring strategy does not include monitoring web content from various geographic locations, it is impossible to know what the end users are experiencing.

In the Web world, both the user interface and the delivery mechanism have been standardized for years, keeping mobile browser compatibility in mind. However, mobile content must be routed to the user through an operator network, and additional operator specific content may be added during the download. Also, different devices render content differently, so when the Web server detects a specific device type it may choose to send a variant of the generic content to the requesting device.

Read More

Web Application Performance Measurement and Tuning

To combat the growing problem of poor web application performance and safeguard the rising amount of business revenue gained via online channels, load testing strategies, tools and services have experienced a transformation in terms of both awareness and adoption.

Business requirements for web application load testing and application performance testing as a means for ongoing performance measurement and tuning have become more rigorous over the past several years.

Measurements derived with load testing tools should provide a clear understanding of where performance bottlenecks reside and aid in infrastructure and capacity planning of computing resources. When derived from meaningful load tests, results serve as a guide to helping IT staff make informed decisions about the performance of their applications and infrastructures.

A solid load testing strategy must complement performance monitoring and analysis in a production environment and, in turn, production monitoring and analysis should be leveraged to improve the accuracy of load tests.

Read More on Enterprise Load Testing for Web Applications.

Challenges And Surprises With Web Load Testing

To gauge a Web site’s capacity and scalability, web load testing is one of the most effective ways. But the load tests need to simulate real scenarios. The huge numbers and ranges of variables involved in Web site load testing will always present challenges and surprises.

A load testing scenario can be made significantly more realistic by simulating the behavior of tolerant and

an intolerant user. Familiarity is a major factor in how quickly a simulated user navigates from one page to the next. As with latency tolerance, different people will behave in different ways: users that are very familiar with the Web site move more rapidly (therefore creating more load per unit of time) than users who are visiting the Web site for the first time and need to read and understand how the Web site is organized to go from one page to the next.

You can also run simple in-house experiments using employees and their friends and family to determine, for example, the page viewing time differences between new and returning users.

Read More on Web Based Testing and Website Monitoring. 

Improving Website Responsiveness Involves Tradeoffs

It is easy for a measurement tool to sit on a server and measure all requests for service—and this kind of measurement has its uses, especially when load testing or investigating bottlenecks. But because of the variety of implementation possibilities, a common problem when measuring RIAs is that related requests may appear to originate from separate units of work on the client.

Correlating seemingly separate measurements with a particular application activity, task, or phase is tricky. The more complex the client/server relationship, especially when it involves concurrent interactions, the harder it becomes for measurement and analysis tools to perform that correlation properly.

Having more design and implementation options also creates new opportunities for developers to make performance-related mistakes. Accidentally or deliberately, developers can implement “chatty” client/server communication styles that perform extremely slowly under some workload conditions. Even with thorough testing, some of these problems may remain undiscovered until after the application is deployed unless applications are subjected to a systematic SLM process that includes measurement
activities to identify, investigate, and fix them.

Read More

To Understand And Measure An Application’s Performance

With RIAs, much of the execution of code in Web applications happens on the browser client, and not just the Web or application server, and therefore understanding what is actually happening inside a real browser has become imperative. To understand and measure an application’s performance, most Web monitoring tools try to emulate a user transaction using measurement technologies that use imitation, or emulated browsers.

These imitation browsers mimic, or emulate browsers like IE and Firefox to simulate a site or application visitor’s behavior. While emulated browser measurements have their place when simply monitoring for availability, a complete Web performance monitoring discipline must include both operational monitoring, primarily conducted using emulated browsers, and true end-user experience monitoring, conducted using real browsers.

Read More

Web Site Preparedness!

One of the major reasons why some companies don’t want to run a web load test on their production site is that they are not looking forward to performing activities such as the database back-ups and restores that a production site load test requires. We believe that such drills are a positive side-benefit of load testing. We have all heard stories of Web sites crashing under heavy load and taking many hours, or even days, to bring their system back.

Practicing system back-ups and recovery following a major load is an extremely important component of Web site preparedness. A load test on your production site will not only show you how well your system can handle a large load, but how well and how quickly your system and crew can recover from a site crash due to overload. If there are problems with post-crash recovery, the right time to discover and fix them is during a test, not while your site is experiencing real traffic peaks.

Read More

Importance Of Website Load Time

An average Web site takes 2– 3 seconds to load. According to Apdex.org, as the load time increases to 4–6 seconds, users start to get frustrated. The level of frustration and the probability of switching to another web site increases if the load time goes beyond 6 seconds. This is where web load testing comes into play and is a very important part of website monitoring.

Many sites do not directly generate revenue, unlike a retail site or travel site, but instead focus on customer service. Sites like credit card, banking, and insurance sites reduce costs, cross-sell products, and help increase customer loyalty by servicing customers online. Website transaction monitoring plays an important role here and is much needed. A poor experience directly translates into increased calls to the call center and a decrease in overall customer satisfaction.  

In the case of online trading sites, online performance is extremely critical. A delay of seconds carries the risk of huge trading losses, and an equally large litigation and regulatory risk. Not only does a technical issue (poor performance, outages, and errors) impact the direct customer or user, it can also have ripple effects.