The Power of Crowd Testing
Contributed by Manoj Narayanan
Cloud-enabled crowd testing fits into the full testing lifecycle when organizations blend traditional and crowdsourcing approaches.
Although crowd sourcing as a concept is not necessarily new, the Internet and mobile devices have acted as catalysts that have triggered a chain reaction, increasing the technique's effectiveness and ROI. Organizations now can leverage easy access to large numbers of individuals – crowds – in achieving business goals.
Where Crowd Testing Applies
Effective testing necessitates analyzing a large number of intuitive as well as esoteric test scenarios. Crowd testing, in many cases, becomes a broad, but effective means of ensuring that all bases are covered. Crowd testing can be leveraged regardless of the underlying development methodology. In fact, it is viable in both agile and waterfall scenarios.
However, crowd testing cannot replace traditional approaches to software quality assurance. If organizations focus only on crowd testing, the quality assurance process loses its scientific rigor and predictability. And if application stability has not been ensured through early lifecycle testing, a vicious loop of ever-increasing defects can ensue.
Thus, organizations need to adopt one of the following strategies for blending traditional and crowd testing approaches to get the maximum ROI.
Navigating the Compliance Journey
However, cloud-based testing poses different challenges than in-house testing.
- As an add-on to testing: Crowd testing can be used as a complementary add-on prior to production release. This will be effective in catching any residual UI and configuration defects.
- For specialized testing scenarios: An example is multivariate testing where end-users select their preferences among specific Web site components or controls. This technique allows testers to determine the highest conversion rates for a site based on numerous simultaneous iterations of A/B testing.
- Leverage based on risk and priority: Use crowd testing for those applications that have a higher Defect Exposure Factor (Number of Releases/yr * Average number of daily critical stakeholder visits) and high business criticality.
- Crowd testing integrated into overall testing, and Testing as a Service (TaaS) strategy: Treat crowd testing as one of a basket of services used in combination with the optimal tool, infrastructure and test type combinations as dictated by the overall test strategy.
Pricing
Current pricing models tend to be a combination of a fixed-based price and a variable component that is dependent on the number of defects uncovered in each run. We have increasingly found that it is more effective for our clients if we bundle this as part of our overall TaaS cloud-based offering.
Future Trends and Looking Ahead
Extinction of pure-play crowd testing seems to be a foregone conclusion. Niche vendors are scrambling to expand their service offerings by entering into alliances with similar players offering specialized testing services such as performance testing. We believe that crowd testing will soon become synonymous with community testing.
We foresee greater integration between social media and crowd test execution. Special interest groups, including those within Facebook and Twitter that are business focused, will help build knowledge and efficient handover across releases even in a crowd testing scenario.
Although crowd testing is fast growing as a viable test option, organizations need to integrate it as part of their overall testing strategy. We offer crowd testing as part of our TaaS platform and believe that this approach can quickly and cost-effectively eradicate defects if properly integrated into the overall testing lifecycle.
Global services players are differentiating by providing more testing services either alone or in partnership with key vendors. Similarly, we have made significant investments and are leveraging our domain knowledge and large testing resource pool to enable us to bridge the gap between crowd testing and community testing.
Read The Power of ‘Crowd’ Testing (PDF) or learn more about Cognizant's testing practice.