In this article we discuss a use case of connecting an IOT device to Zetaris for further analysis.
Technical Diagram:
Prerequisites:
-
A Zetaris Instance
-
A Mobile device with the ability to connect to Azure IoT Central
Connectivity Guide:
Part 1: Connecting your Mobile Device to an Azure IoT Central Application
In your Smart Phones application store, i.e. Google Play Store, Download the Azure IoT Plug and Play app. Link can be found in the following Microsoft document.
Follow the Steps from the “Connect the app” section in the following Microsoft documentation.
Use your smartphone as an Azure IoT device
Part 2: Exporting IoT Stream to Azure Event Hub Entity
Creating Azure Event Hubs
Within the Azure Portal, search for “Event Hubs” in the search bar. and select Event Hubs as shown in the screenshot.
Select Create and input your desired values. Please refer to the below screenshot for an example of inputs.
Once created navigate to your created Event Hub resource and select Event Hubs under Entities. Create a new Event Hub Entity using the [+ Event Hub] Button as shown in the screenshot.
This Event hub will contain the connection details used to connect to Kafka.
Once created select your Event Hub Entity and navigate to Shared access policies. Create a new policy with Send and Listen privileges. Once created, select the newly created policy and copy the Connection string-primary key into notepad. This will be used later on.
Setting up Azure IoT Central to export data.
Navigate to your Azure IoT central url in your browser and select your application.
Navigate to Data Export in the side navigation bar as shown in the following screenshot.
Select Destinations and Create a new Destination and input the following information.
|
Destination type |
Azure Event Hubs |
|
Authorization |
Connection String |
|
Connection String |
Copied Connection from previous step |
Once you have validated the information click on the save icon.
Navigate to Exports and create a new export.
Within the Data section select “Telemetry” for “Type of data to Export”. Within Destinations select your previously added Destination and click on Transform. In build transformation query change it to the following.
. as $in | .telemetry[] | {
deviceId: $in.device.id,
messageSource: $in.messageSource,
enqueuedTime: $in.enqueuedTime,
sensor: .name,
sensor_data: .value,
sensor_data_x: (.value.x // null),
sensor_data_y: (.value.y // null),
sensor_data_z: (.value.z // null),
component: $in.component
}
Click on Update.
Click on the save icon.
Part 3: Exposing your IoT stream through Kafka
Create a Kafka Virtual Machine:
Navigate to Azure Marketplace and search for Kafka. Install the following Kafka in your desired resource group.
Create the Exposer:
SSH into the VM and clone the following repository.
git clone <placeholder>.git
Install the Maven dependency.
sudo apt install maven
Update the consumer.config file and change the following:
|
NAMESPACE |
Your Event Hub Namespace |
IoT-EventHub |
|
{YOUR.EVENTHUBS.CONNECTION.STRING} |
The connection string you copied to notepad |
|
Run the following commands to expose your event hub
cd ~/event-hub-to-zetaris
mvn clean package
mvn exec:java -Dexec.mainClass="EventHubToZetaris"
Part 4: Connecting your IoT Stream to Zetaris
Connect your Zetaris Instance in your browser and navigate to the Schema Store View feature tab.
Copy the following SQL into your SQL Editor and change to your respectively.
REGISTER STREAMING DATASOURCE iot_telemetry INPUT_SOURCE KAFKA INPUT_FORMAT JSON INPUT_OPTIONS(
kafka.bootstrap.servers "<Kafka VM Public IP>:9092",
kafka.topic "telemetry"
)
INPUT_SAMPLE
{
"component":"sensors",
"deviceId":"xxxxxxxxxxxxx",
"enqueuedTime":"2023-01-12T05:52:23.735Z",
"messageSource":"telemetry",
"sensor":"accelerometer",
"sensor_data":"xxxxxx",
"sensor_data_x":0.11492519080638885,
"sensor_data_y":0.8882759809494019,
"sensor_data_z":9.759063720703125
};
REGISTER STREAMING AGGREGATION full_telemtry_data FROM iot_telemetry OUTPUT_MODE APPEND OUTPUT_FORMAT PARQUET OUTPUT_OPTIONS(
path "/home/zetaris/customer-data/streaming/iot_telemetry/telemetry_data"
)
AS
select timestamp, json_body.* from iot_telemetry;
Notice it gets listed as a Zetaris streaming source
