PodChats for IoT: Working with IoT data analytics – Technologist

By 2025, it is expected that IoT devices will generate roughly 73.1 Zettabytes of data. Around that time, 30% of all data will be real-time, with IoT accounting for nearly 95% of it, 20% of all data will be critical and 10% of all data will be hypercritical.

The real-time nature of IoT data presents opportunities, challenges and threats to organisations. Analytics will have to happen in real time for companies to benefit from these types of data.

According to Niraj Naidu, APJ head of field engineering for DataStax, the internet of things (IoT) refers to the billions of internet-connected devices around the world that use different types of sensors to collect real-time data remotely.

He adds that the IoT use cases span many different industries in a wide range of applications, depending on the application, the data that is collected from these devices can be used to perform analytics.

These devices need support from a modern technology stack that can ingest and handle a continuous flow of real-time and often time series data. Some IoT devices may have multiple sensors collecting different types of data.

“All that data flows in and is integrated into a platform where analytics are applied, patterns emerge from the analytics and insights are uncovered. These insights are then effectively converted into useful information that is delivered to end users via the IoT application user interfaces.”

Niraj Naidu

“And so, we see the type of valuable information will vary by each application’s purpose, but it could be a recommendation it could be an alert, a status update, or any other variety of use cases being tackled by specific IoT initiatives that you know, these organizations have,” he added.

The six challenges of a growing IoT ecosystem

Naidu forecasts that by 2024, the global IoT market is predicted to surpass a trillion dollars annually.

He posited that due to this growth, companies are now speedily accumulating hundreds of terabytes or even 1000s of terabytes of data, depending on the industries that they’re based in.

“These companies are therefore having to deal with many different challenges when it comes to managing data produced by these IoT devices or even gadgets.”

There are six main things that we tend to see:

Scalability: Companies need a modern architecture that can quickly and seamlessly scale alongside the number of users’ devices or even the associated data volumes if they suddenly skyrocket to this.

Security: Data flowing in from IoT devices can be located anywhere globally, and there is a high probability that some if not all of it, is sensitive information. A company trying to completely handle the security of this data internally places itself at risk of a breach occurring.

Control: Data should be a company’s most asset, and IoT collection and analysis can yield some amazing insights that will lead to them creating a competitive advantage for them in their markets. Organizations should therefore have complete control over that data and their data. And be able to migrate whenever and wherever they want and make it a first-class citizen.

Performance: The value that can be gained from IoT data depends on its timelessness. And the effectiveness of IoT devices largely relies on the responsiveness of the system. The platform that can truly handle millions of devices, and elastically scale with any unexpected spikes, can do all of this without slowing things down

Flexibility: Companies need that flexibility to speedily adjust, when necessary, to handle that rarity of data and devices. The database should have the ability to deal with many different types of data models and cloud environments that these IoT devices are generating.

Availability: That’s one of the other key challenges that organizations are having and the success of an IoT system requires continuous flow and exchange of data, and as such going offline, it could be disastrous.

“When it comes to smart products, smart devices customers expect to access and information to be just to be available right at the click of a finger. The systems must therefore always be online, with no single points of failure,” he concluded.

Data infrastructure for IoT

Naidu suggests that the data infrastructure for an IoT system today needs to be open. It is built on a modern data management layer that once again can ingest large volumes of high-velocity data that these IoT devices and gadgets are created.

There are three parts to the system there is device connectivity, IoT Hub and the business layer.

Device connectivity is metadata information for each deployed device that needs to be managed. Each of those management layers is looking at specific things like the device registry device. Once again, metadata right device configuration, looking at device states, the device commands and interactions, and device shadows.

This part of the IoT system also contains field gateways, specialized devices, or software that acts as a communication enabler in even a local device control system, and a device data processing hub.

The IoT Hub is where all the data comes together to allow operations, administration, and insights to take place. It has two or more types of storage, looking at a hot layer for ingesting and a holding layer for recent data and another cold layer for older data.

That hub can be on-premises or close to the edge or in the cloud depending on the scale of the data or latency requirements that the organization may have.

There are then two types of analytics that can be conducted via that hub. There’s real-time analytics right as data arrives at a central hub. It is streamed with event streaming technologies, which allows complex event processing tasks and analytical tasks to take place.

Batch analytics is ideal for cases where large amounts of data must be analysed right with batch queries, or even ad hoc queries, as those systems or users require.

At the business layer is where all the data from those devices’ gateways and, other sources come together to provide analysis for actionable insights. This analysis provides the ability to spot anomalies and explore trends and measure operational efficiency is just an example.

Other tools like artificial intelligence and machine learning form the basis for predictive maintenance and operation.

“I think that those three elements, they’re aligned around device connectivity, the IoT hub in the business layer is what really kind of help form the right data infrastructure, architecture for IoT.”

Click on the PodChat player as Naidu elaborates on how to achieve better results using IoT analytics.

1. What is IoT analytics?
2. How prevalent is the use of IoT analytics in Asia?
3. We know that IoT produces a lot of data. What are the real challenges why enterprises struggle to better utilise/monetise the data produced by IoT?
4. Can you name a high-value/promising use case of IoT analytics?
5. What would the data infrastructure for IoT look like?
6. Do you see IoT streaming data as further complicating (or facilitating) IoT analytics adoption?
7. Where does DataStax sit in the adoption of IoT analytics?