Taking Testing to the Cloud

Testing is an especially good use of cloud computing. Here's a look at the benefits you can expect (beyond cost-savings) and the pitfalls you should watch out for.

Cloud computing has opened up new opportunities for testing departments. Testing has traditionally required expensive dedicated infrastructure and resources that were only used sporadically. The growing complexity of business applications also made it harder to build and maintain in-house testing facilities that mimic real-time environments.

Compared to maintaining an in-house test environment cloud-based testing offers lower costs, more flexibility enhanced collaboration and several other benefits. CIOs and CTOs should move carefully, however, as they explore challenges such as data security and a lack of standards.

Source: IDC Survey in 2009

Advantages of Cloud Testing

Cost is, of course, a major advantage of moving testing to the cloud. It frees companies from large capital expenditures for creating test environments, and pay-as-you-go models mean they don't pay for idle test environments. But it's important to note that cloud-based testing offers other meaningful benefits.

• Support for complex apps: Applications are increasingly becoming dynamic, complex, distributed and component-based. In-house test environments are proving to be increasingly expensive and ineffective when handling these complexities.
• Improved test quality: Roughly 30% of defects are attributable to inaccurate configuration of test environments. Cloud-based testing service providers offer a standardized infrastructure and pre-configured software images that are capable of reducing such errors considerably.
• Time-to-market: Creating on-premise test environments can be time-consuming and cause project delays. With cloud-based testing, organizations no longer need to wait to buy and configure servers, or to license and install programs and testing tools.
• Real-time testing: Cloud-based test environments allow testing teams to analyze application performance and look for bottlenecks even while tests are running.
• Scalability: The cloud allows testers to quickly and easily scale from thousands to millions of concurrent users to assess an application's breaking points and capacity thresholds under highly unpredictable demand.

Operational Challenges for Testing in the Cloud

However, cloud-based testing poses different challenges than in-house testing.

First, there are currently no universal or standard solutions to integrate public cloud resources with users' internal data centers. Each cloud provider has their own architecture, operating model and pricing mechanisms and offer very little interoperability. This poses a challenge if companies need to switch vendors.

Security is another major concern, mostly because data and code may be stored in a remote location beyond an organization's legal and regulatory jurisdiction. Yet another challenge is that some cloud providers offer only limited types of configurations, technology, servers and storage, networking and bandwidth, making it difficult to create real-time test environments.

Improper choice of cloud-based use and pricing options is another risk. While some vendors offer pay-as-you-go services, they are only cost-effective when the right plan and servicer provider are chosen for the anticipated needs (e.g. space vs. RAM vs. bandwidth). Costs can quickly spin out of control if resource estimates differ wildly from actual usage.

Also be aware of associated expenses, such as the cost of encrypting data to assure its security in the cloud. And remember that cloud disaster recovery isn't necessarily built in just because the test infrastructure is in the cloud. Test teams should also rigorously plan their test environments, test infrastructure and arm themselves with security and greater control over data.

To gain confidence, create a proof of concept. Ask yourself what cultural and process changes are required to move to cloud-based testing, who owns service management, and what changes in organizational/financial process alignment must be made to manage/provide for a new service.

Start small and gain confidence in the benefits of cloud-based testing. Cost of operation and ownership will fall over time in companies that intelligently embrace pay-as-you-go or on-demand services. Use an experienced partner that can ensure cost reductions and faster time-to-market. Companies that start early and push forward will reap first-mover advantages that far outweigh the pioneering risks.

Read the complete white paper Taking Testing to the Cloud (PDF) and learn more about Cognizant's testing services and cloud-based services.