This article covers the characteristics of IoT analytics and the challenges faced when adopting it in the industry. It also presents real world use cases of IoT analytics.
The standard definitions of IoT are:
- “A network of physical objects or things accessed through the internet, as defined by technology analysts and visionaries. These things contain embedded technology to interact with internal states or the external environment. In other words, when things can sense and communicate, it changes how and where the decisions are being made and who makes them.” – Cisco
- “The IoT can be viewed as a global infrastructure for the information society, enabling advanced services by interconnecting (physical and virtual) things based on existing and evolving interoperable information and communication technologies.” – ITU
- “An IoT system is a network of networks where, typically, a massive number of objects, things, sensors or devices are connected through communications and information infrastructure to provide value-added services via intelligent data processing and management for different applications (e.g., smart cities, smart health, smart grid, smart home, smart transportation, and smart shopping).” — IEEE Internet of Things Journal
Overall, the Internet of Things (IoT) is a network of devices which can sense, accumulate, and transfer data over the internet without any human intervention. IoT and data remain intrinsically linked together. Data consumed and produced keeps growing at an ever-expanding rate.
Internet of Things (IoT) analytics assesses the wide range of data collected from IoT devices. It analyses vast quantities of data and produces useful information called Big Data. It addresses large, diverse, complex, longitudinal, and/or distributed data sets generated from instruments, sensors, internet transactions, emails, videos, click streams, and/or all other digital sources available. The processing and integration of this heterogeneously generated data is done by IoT analytics.
IoT analytics helps gain insights into the operations of an enterprise, boosts productivity, increases the quality and flexibility of operations, and minimises material consumption.
Industry adoption of IoT
According to Gartner, more than 30.9 billion IoT devices will be in use worldwide by 2025. A forecast by International Data Corporation (IDC) estimates that there will be 41.6 billion IoT devices in 2025, capable of generating 79.4 zettabytes (ZB) of data.
The global Internet of Things market is estimated to rise from US$ 478.36 billion in 2022 to US$ 2465.26 billion by 2029 at a CAGR of 26.4 per cent during the forecast period, as per Global Wire.
According to McKinsey, by 2030, the IoT market will be valued between US$ 5.5 trillion and US$ 12.6 trillion globally, including the value captured by consumers of IoT products and services.
Key characteristics of IoT analytics
IoT analytics integrates business intelligence and data analytics, including conventional and Big Data. Its characteristics are explained briefly below.
Volume: The volume of data processed by IoT data solutions is substantial and ever-growing. High data volumes demand massive data storage, data processing, data preparation, data curation, and data management processes. Big Data requires processing high volumes of data, that is, data of unknown value — for example, network traffic, sensor-enabled equipment capturing data, and many more. Some of the data sources that are responsible for generating high data volumes are:
- Scientific and research experiments
- Sensors, such as temperature, pressure and GPS sensors
- RFIDs
- Smart meters and telematics
Velocity: In IoT data analytics environments, massive data gets accumulated in a short time, as sensors collect data pretty quickly. The velocity of data arrival within an enterprise’s perimeter is directly proportional to the time taken to process the data. It demands highly elastic, highly available data processing and massive data storage solutions for the enterprise.
Variety: IoT data is primarily unstructured or semi-structured. Such data types require additional processing to derive meaning and the supporting metadata. Further complexity arises when data from a known source changes without notice.
Veracity: Veracity refers to the quality or fidelity of data. Assessment of data that enters IoT data environments for quality can lead to data processing activities to resolve invalid data and remove noise. Data acquired in a controlled manner usually contains less noise than data acquired via uncontrolled sources. Data with a high signal-to-noise ratio has more veracity than data with a lower ratio.
Value: There is a range of quantitative and investigative techniques to derive value from data. The cost of data storage and computing has fallen exponentially, thus providing an abundance of data for statistical sampling and other techniques.
Response time and reliability: The essential requirement of IoT data analytics is better response time and reliability. The solution is built on three fundamental factors: deterministic analytics, data computation, and quick response time.
Bandwidth: The massive amount of data generated by IoT devices is gathered from control systems. Handling of this data requires better bandwidth. The increased network and infrastructure required to transfer data from one domain to another is compensated by creating valuable insights.
Security: Transferring raw data outside of controlled areas comes with security concerns and the associated costs. It may be more efficient to perform data analytics locally and share the necessary summaries with other domains.
Analytics maturity: Analytics involves the processing of raw data (measurements) into descriptions (information), and then contextualizing the results (knowledge) and benefiting from historical experience (wisdom). The maturity of the processed raw data depends on the design of the analytics solution.
Temporal correlation: Applying analytics closer to where the data is generated reduces the burden of correlation. It helps IoT systems to correlate the data between multiple sensors and process control system states.
Provenance: Performing analytics at a lower architecture tier maintains the genuine sources of the data as it progresses through the IoT layers.
Principles of IoT analytics
Architecture principles provide a basis for decision-making when developing IoT data analytics solutions and design. They form a structured set of ideas that collectively define and guide solution architecture development, harmonising decision-making across an organisation.
The principles that guide enterprises in the way they approach IoT analytics data are:
Data is an asset: Data is an asset that has a specific and measurable value for the enterprise.
- Data is shared: Data must be shared across the enterprise and its business units. Users have access to the data that is necessary to perform their activities.
- Data trustees: Each data element has trustees accountable for data quality.
- Common vocabulary and data definitions: Data definition is consistent, and the taxonomy is understandable throughout the enterprise.
- Data security: Data must be protected from unauthorised users and disclosure.
- Data privacy: Privacy and data protection is considered throughout the life cycle of a Big Data project. All data sharing conforms to the relevant regulatory and business requirements.
- Data integrity and the transparency of processes: Each party to a Big Data analytics project must be aware of and abide by their responsibilities regarding the provision of source data and the obligation to establish and maintain adequate controls over the use of personal or other sensitive data.
Key challenges of IoT analytics applications
Heterogeneity: IoT data streams tend to be multi-modal and heterogeneous in their formats, semantics, and velocities. To access IoT data sources, database drivers and adaptors need to be deployed in Big Data applications. These applications incorporate features like interoperability, transaction management, and common data formats to address the heterogeneity.
Data quality: Different manufacturers provide various sensor categories and types. Substandard and unreliable sensors generate noisy and incomplete IoT data streams. Usage of these data streams leads to uncertainty and inaccuracy in IoT analytics. To improve the data quality and eliminate the noise, statistical models and probability based applications are developed to feed into IoT data analytics applications to generate an accurate output.
Real-time data sets: IoT devices provide high-speed and real-time data. In the Big Data ecosystem, data streaming platforms consume real-time data sets to generate valuable insights.
Location and time: IoT data comes with temporal and spatial metadata information, directly associated with its business value in each application context. These spatial and temporal dimensions can serve as a basis for dynamically selecting and filtering streams towards analytics applications for specific timelines and locations.
Security and privacy: IoT applications collect and process personal data and device data. IoT analytics helps in exhibiting strong IoT security requirements and managing privacy sensitivities. The data generated by IoT data analytics needs to be encrypted, secured, and stored.
Data processing: IoT data analytics systems need to follow the steps listed below for data processing:
- Collection of data aggregated from a variety of sources, in a range of formats, and at multiple frequencies
- Understanding the data and categorising it
- Preparation of data
- Processing data from different sources
- Storing the information in time series for analysis
- Analysing the results
- Building applications to fulfil business processes
IoT analytics architecture
The requirements for an IoT analytics system with respect to its architecture are as follows.
- Context: Capture the context at all times, 24 hours a day, 365 days a year.
- Standards: Leverage standard protocols of communication between IoT devices and enterprise systems.
- Scalability: Always respond even during increased load; increase capacity proportionally as resources are added.
- Data management: Efficiently manage enormous volumes of data.
- Connectivity: Provide high network connectivity for large data payloads and continuous streaming.
- Security: Move and encrypt information securely even as IoT introduces new risks and vulnerabilities.
- Interoperability: Network all systems together and ensure interoperability of all data.
- Device management: An ability to connect to devices remotely at scale to manage them such as updating security credentials and firmware updates.
- AI-driven analytics: Enrich and explore prescriptive and predictive analytics data to deliver actionable insights.
Figure 1 shows the logical architecture of the IoT analytics system with key components and layers. While there exist many standard logical architectures for IoT analytics, this one is based on my practical experience across various domains and technologies.
The various layers for an enterprise application are classified as:
- Data source layer
- Edge layer
- Analytics layer
- Application layer
Data source layer: This layer consists of IoT devices and IoT gateways. Sensors and actuators are essential components of IoT.
Sensors sense the environment to monitor and detect changes of a particular attribute and collect the data, such as a change in temperature, heat, or cold. The output of the sensor is an electrical signal.
Actuators are devices used to control the system. They receive electrical output from sensors and produce a physical change. For example, if the sensor detects a change in temperature in a room, the actuator can turn a device on or off to maintain and control it.
These sensors and actuators establish a link between the physical and digital world via an IoT gateway. This layer basically deals with the gathering of specific data from data sources. This data is then passed to the edge layer so that it can be securely transmitted to the analytics layer.
Edge layer: The edge layer is an entry point for all things or devices to capture and process the data. The layer consists of a data store, services to analyse the data streamed from the IoT devices and execute AI-driven analytics locally in real-time, security mechanisms (including data encryption), and control applications.
It consists of an IoT gateway that facilitates bi-directional communication with the platform and enterprise systems. It also implements edge intelligence with different processing capabilities, enabling connectivity between devices, routing, filtering, and protocol and identity translation.
Analytics layer: The primary purpose of this layer is to interface with the edge layer to receive, store, process, and analyse the data. In addition to data acquisition capabilities, it offers AI-driven analytics for intelligence and visualisation, device management, security, connectivity and network management, and operations.
The data from devices is collected and aggregated from the IoT device gateway. Real-time AI-driven analytics and visualisation combined with machine and deep learning models are applied to the data to derive intelligence.
Machine learning is also performed over the collected device telemetry data, and this data is blended with other enterprise data or secondary sources of telemetry from other systems or organisations.
Application layer: This layer acts as an interface to interact with applications for end users. Devices like smartphones, computers and tablets are used as a medium to make IoT applications interactive.
Real world use cases of IoT analytics
Implementation of IoT analytics based solutions helps enterprises eliminate redundant assets and products, leading to better operational efficiency and cost savings.
IoT for digital government: The vision of any government in the digital era is to develop connected and intelligent IoT based systems that contribute to the economy, improve citizen satisfaction, and lead to a safer society and environment sustainability.
Processing of data feeds from various sources like weather and environmental sensors, traffic signal cameras, parking zones, etc, leads to improved government-IoT agency coordination and better services to citizens.
IoT for manufacturing: IoT in manufacturing allows richer insights in real-time. It can facilitate the production flow in a manufacturing plant, as IoT devices automatically monitor development cycles and manage inventories.
IoT for supply chain management: Manufacturers use IoT to monitor their supply chain and get estimates of the available resources. This includes information regarding the status of the work on the floor, equipment collection, and the delivery date of required materials.
IoT for smart homes: IoT helps to analyse and optimise use of devices in a smart home. This can mean simple switching off or dimming of lights, and modifying multiple home settings to optimise energy use. It also helps discover problematic energy consumption by older and damaged appliances, or faulty system components.
IoT for media: IoT based applications in media and advertising lead to a customised experience in which the system analyses and responds to the needs and interests of each customer. It helps in understanding the customer behaviour patterns, buying habits, preferences, culture and other characteristics.
IoT for healthcare: IoT based applications/systems of healthcare enhance the traditional technology used today. These devices help increase the accuracy of the medical data collected from a large set of devices connected to various applications and systems. They also help gather data to improve the precision of the medical care delivered through sophisticated integrated healthcare systems.
- IoT devices give direct 24/7 x 365 access to the patient in a less intrusive way. IoT based analytics and automation allow health providers to access patient reports before they arrive at the hospital. They also help improve responsiveness in emergency healthcare.
- IoT-driven systems are used for continuous monitoring of a patient’s health status. These employ sensors to collect the physiological information analysed and stored on the cloud. Doctors access this information for further analysis and review. This way, a continuous automated flow of information is provided.
- Patients’ health data is captured using various sensors, analysed and sent to the medical professional for proper remote medical assistance.
IoT for education: IoT customises and enhances education by optimising content and forms of delivery.
- Student tracking: IoT facilitates the customisation of education to give students access to what they need.
- Instructor tracking: Educators can use the IoT platform for customised instructional designs for their students.
- Facility monitoring and maintenance: IoT based applications improve the professional development of educators.
- Data from other facilities: IoT also enhances the knowledge base used to establish education standards and practices. It introduces large, high-quality, real-world data sets into educational design.
IoT for agriculture: IoT minimises human intervention in farming analysis and monitoring. IoT based systems detect changes in crops, the soil, etc.
- Crop monitoring: IoT based devices can be used to monitor crops and the health of plants using the data collected from them. These devices can also monitor pests and diseases proactively in the early stages of infestation.
- Food safety: IoT helps connect the entire supply chain for ensuring the food is safe when it reaches the customer.
- Climate monitoring: IoT based devices are used to monitor temperature, humidity, the intensity of light, and moisture of the soil. The data is ingested into the central repository to trigger alerts and automate water, air, and crop control.
- Logistics monitoring: Location based IoT devices can be used to track vegetables and other farm products during transport and storage.
- Livestock monitoring: Farm animals can be monitored using IoT devices to detect potential signs of disease. The data can be analysed from the central repository, and the farmers can share relevant information.
IoT analytics has a number of benefits that are helping transform businesses and different industries.
- Greater transparency; transaction histories are becoming more transparent
- Increased visibility and better control, resulting in quicker decision-making
- Flexible scaling of business needs and expansion in new markets
- Reduction in operational costs due to automation and better resource utilisation
- New revenue streams, resulting from the clearing of operational challenges
- Enhanced customer experience based on an analysis of the purchase history
IoT is one of the most significant sources of Big Data, which is rendered useless without analytics power. IoT analytics comes into play when voluminous amounts of data needs to be processed, transformed, and analysed at a high frequency. It helps to provide advanced services across many domains.
| Acknowledgements |
| I would like to thank Santosh Shinde of BTIS, enterprise architecture division of HCL Technologies Ltd, for giving the required time and support while writing this article as part of architecture practice efforts. |
