According to Gartner, the Internet of Things installed base will grow to 26 billion units by 2020. Data analytics from this evolving connected world holds immense promise. But often lost amid the IoT excitement is the paradigm shift it entails for the quality assurance (QA) function, which needs to embrace the technology changes in the products and services offered in the IoT's broadening spectrum.
Defining QA's Role
A plethora of applications is under development that will soon expand the existing hyperintelligent, connected world. Organizations need to perform testing on these applications before they reach prime time. The role of QA is critical, because it involves testing hardware and software and transmitting massive amounts of real-time intelligence. Complicating things is the IoT's lack of standards. Top challenges for QA managers include:
Hardware/software convergence: QA needs to shift away from sheer testing of applications in a defined environment, to testing the experience in a dynamic environment.
The insufficiency of a "working system": To certify an IoT system or service, it is not enough to merely ensure a working set of device software; accurate product delivery must be the objective. This requires a robust QA validation process.
Large number of sensor interactions: It is a challenge to create an environment for testing a real-time IoT implementation. Understanding application intelligence and the devices' real-time complexity constitutes an extremely new approach requiring both art and science techniques that are unfamiliar to many application developers and QA experts.
Addressing IoT's QA Complexity
A comprehensive QA strategy is required to cover the depth and breadth of IoT testing. Such a strategy must encompass the different types of testing, test lab setups, testing tools and simulators that are needed. Here is a partial list of the required elements, beyond what is needed for traditional software testing:
Conformance with standards: The device's performance attributes, specific to devices and sensors, must be validated.
Interoperability: Different devices need to support required functionality among themselves, other external devices and implementations.
Security: As the raw number of sensors rises exponentially, so does the number of nodes on the network, increasing the areas that are vulnerable to potential security breaches. Therefore, it's crucial to tackle data privacy and security concerns across the IoT ecosystem.
Network capability and device-level tests: The specific aspects of network communication such as connectivity need to be validated by simulating different network modes, in addition to device-level validation, such as energy consumption tests, etc.
Usability and user experience: Usability and user experience are highly important; this involves human/machine interaction and real-time capabilities.
IoT services and the back-end IoT environment: While integration testing of the interfaces is key, underlying data complexity can be challenging.
Creating a QA environment to enable validation of such an interface means addressing the growing data volume, velocity and variety challenges of the IoT ecosystem. For example, testers can leverage sandboxes of development services or create mock environments using virtualization tools.
IoT Testing Types
A simple IoT ecosystem borrows several software testing approaches from traditional QA methods to validate IoT applications. Testers must focus on all three core components of the IoT system – the "things" themselves, the communication infrastructure and the computing infrastructure – to enable functional and connectivity testing. Testing must also address:
Performance testing, which covers the rapidity of the communication network model, as well as the internal computation capabilities of the embedded software system.
Security testing, which covers privacy, autonomy and spying.
Compatibility testing with the numerous possible combinations of device, protocol and mobile OS versions.
Exploratory testing to test from the user's perspective and beyond predefined test procedures.
Solutions and Framework Opportunities
The IoT ecosystem introduces myriad QA challenges, which the QA organization can view as opportunities to build frameworks and solutions, including:
Protocol simulators, which can help when a wide variety of device endpoints and interfaces needs to be validated.
Data recorders, which can be used to validate across device sets.
Virtualization, which enables faster turnaround and reduced costs due to minimal dependency on the production environment for testing. It also leads to faster identification of defects, thereby establishing a new dimension within the validation process.
With ever-greater blending of the IoT into business and IT, QA teams must upskill beyond traditional functional testing and gear up for integrated testing of embedded software, IT solutions and big data – and understand their interdependence on one another. To prepare, QA organizations should focus on the following:
Orient staff to gain a mix of skills, combining QA, quality engineering and hardware validation.
Build collaborative QA teams with a view toward performing integrated tests spanning hardware, software and big data layers, thereby augmenting the niche and broader aspects of IoT testing.
Look for tool-build opportunities. As the distinctions between hardware and software blur, a plethora of opportunities are available to build solutions that enhance QA across the internal systems ecosystem.
Build labs that serve the entire digital portfolio, to experiment and simulate real-time experiences that inform smarter ways of testing.
Build a culture of "test as a user" vs. merely testing against requirements; this will ensure a well-established "experience" component of the IoT stream for a comprehensive quality product or service.
For more about leveraging QA for the IoT, read our whitepaper The Internet of Things: QA Unleashed [PDF]. Visit Cognizant Digital Works for more on digital experiences and the IoT section of our website to learn more about how your organization can optimize its use of the IoT. Or visit our Quality Engineering & Assurance practice.