Mobile Web: Load Testing and Ongoing Performance Monitoring

Many retailers are making efforts to enable genuine mobile commerce, as opposed to trying to drive traffic from mobile to other channels to complete the transaction. And many are bringing their mobile development efforts in-house, after testing the waters with a third-party solution. Both of these scenarios involve significant risk, making it all the more imperative to perform robust load testing and ongoing performance monitoring.

The big challenge that mobile sites face that’s different than the desktop Web — in addition to the inherent slowness of over-the-air (OTA) signals — is the plethora of devices, operating systems, and carriers. A mobilesite or app has to be vetted in all the major configurations that ultimately control what is displayed in the user’s hands.

Any outage or excessive slowness during peak holiday periods can inflict serious pain on a retailer’s bottom line, whether the retailer is big or small, online-only or hybrid. The only way to make sure a website can handle an extraordinary rush of traffic is to thoroughly test it well in advance and every time changes are made. The most solid test regimens subject a site to traffic that’s multiples of the highest demand projected by the sales and marketing department.

Read More

Real tests use real devices

It's not that testing with real devices is the gold standard, though that's one way to look at it. The fact is, there is simply no way to emulate the behavior of a mobile app in the field unless the device is literally in the field. System overhead, memory usage, CPU speed — a host of variables impact app functionality in ways that just can't be reproduced in a lab. To get real test results, you have to deploy real devices.

But creating an internal test program using real devices is a challenge. The devices have to be bought, carrier contracts have to be established, test scripts created, users trained, results compiled and analyzed — for each geographic market. And then the devices and contracts have to be dealt with after the testing is done, or maintained for ongoing monitoring.

This complexity is why so many enterprises use an outside test partner that offers an established infrastructure with hundreds or thousands of devices deployed over a broad geography. The test provider works with the client company to develop the necessary scripts, and then leases time to them on its network to run the tests. This is testing in the "public cloud," which means that many clients utilize the same devices and infrastructure to conduct their tests. It's an ideal solution for most companies — there's no upfront capital expenditure and tests can be quickly executed on demand, on a budget-friendly pay-per-use basis.

Source: http://keynote.com/benchmark/mobile_wireless/article_mobile_app_performance.shtml