Automate IoT Infrastructure Creation

Introduction

This is the second article in our IoT series. The first one covered the development of a temperature / humidity sensor as an edge device. The standalone sensor is only accessible via REST from a local WiFi. Furthermore, we just see the latest value. There are no analytics or interactive visualization. To resolve these problems, this article describes the automated creation of appropriate cloud infrastructure to extend the sensors functionality:

• Cloud services for users to visualize, analyze and process the sensor data
• Ansible for an automated setup and to document the infrastructure

We will use the following network topology:

Every sensor will push its data into a time/series InfluxDB. The database is added as a data source into Grafana. It acts as a frontend for users to configure alarms, visualize or analyze the data. Both services are running as docker containers.

Source Code

The projects source code is available here on Github. We also created two standalone examples for either InfluxDB or Grafana.

InfluxDB

InfluxDB is a open-source time/series database. It provides a REST interface, web access and a SQL-like query language. It uses the following logical data structure:

• Organization: Workspace for a group of users.
• Bucket: Belongs to an organization and has a retention policy.
• Measurement: Belongs to a bucket and can be compared to a SQL table.

InfluxDBs default port is 8086 which can be used to write data via REST API:

#!/bin/bash

temp="$((20 + $RANDOM % 6))"
hum="$((40 + $RANDOM % 11))"
echo writing tuple temp==$temp and hum==$hum into db

curl --request POST \
"http://localhost:8086/api/v2/write?org=phobos&bucket=cloudsensor&precision=ns" \
  --header "Authorization: Token $1"\
  --header "Content-Type: text/plain; charset=utf-8" \
  --header "Accept: application/json" \
  --data-binary "
    office temperature=$temp,humidity=$hum
    "

The example above writes (almost) random temperature and humidity values into the database with curl and POST. Organization and bucket names are passed via URL parameters. The measurement name is the first element in “data-binary” and called “office” in this example. Since we do not specify a timestamp, it is appended by InfluxDB automatically. A token is used to authenticate. It can be achieved via web access or by directly opening a shell into the container:

#!/bin/bash

token=`docker exec influxdb2 influx auth list --json | jq -r '.[0].token'`
echo $token

We have extended the ESP32 C source code of our temperature sensor to write its data into an InfluxDB database. It is pretty much straight forward:

//create header
char auth_header[150];
snprintf(auth_header, 150, "Token %s", CONFIG_CLOUDSENSOR_INFLUX_TOKEN);
esp_http_client_set_header(client, "Authorization", auth_header);
esp_http_client_set_header(client, "Content-Type", "text/plain; charset=utf-8");
esp_http_client_set_header(client, "Accept", "application/json");

// POST with temp/humidity in body
char post_data[64];
snprintf(post_data, 64, "%s temperature=%.1f,humidity=%.1f", 
   CONFIG_CLOUDSENSOR_NAME, temp, humidity);
esp_http_client_set_method(client, HTTP_METHOD_POST);
esp_http_client_set_post_field(client, post_data, strlen(post_data));

ESP_LOGI(TAG, "Sending to influx db: %s", post_data);
esp_err_t err = esp_http_client_perform(client);

The functions esp_http_client_set_header() and esp_http_client_set_post_field() are invoked to configure a header and a body in the HTTP request while esp_http_client_perform() sends humidity and temperature to an InfluxDB instance.

Grafana

The Grafana web application will be used as an InfluxDB frontend. For proper usage, we have to configure

• an InfluxDB data source which acts as the glue between InfluxDB and Grafana
• a dashboard which queries the previously mentioned data source and visualizes the data accordingly

Data Source

We will use Grafanas REST API running on port 3000 and a Python script to add an InfluxDB data source. The corresponding source code is available here. Two REST nodes are important:

• /login to authenticate via user name and password
• /api/datasources to add influx as a data source

The following code snippet demonstrates the InfluxDB / Grafana integration:

datasources_post = session.post(
       os.path.join(grafana_url, 'api', 'datasources'),
       data=json.dumps({
           'access': 'proxy',
           'database': '',
           'name': f'{GRAFANA_DATASOURCE}',
           'type': 'influxdb',
           'isDefault': True,
           'url': f'http://{influx_host}:{INFLUX_PORT}',
           'user': '',
           'basicAuth': False,
           'basicAuthUser': '',
           'basicAuthPassword': '',
           'withCredentials': False,
           'jsonData': {
               'defaultBucket': f'{influx_bucket}',
               'httpMode': 'POST',
               'organization': f'{influx_organisation}',
               'version': 'Flux',
           },
           'secureJsonData': {
               'token': f'{influx_token}'
           }}),
       headers={'content-type': 'application/json'})

Obviously, type is set to influxdb. Since we are using a token to authenticate with influx, the whole user, basicAuth, basicAuthPassword, etc. entries remain empty. The element jsonData contains the bucket and organisation name. The token is passed as a part of an secureJsonData structure to make sure it is encrypted in Grafanas database.

Dashboard

After the data source is up and running, a dashboard should be created to access temperature and humidity values. During dashboard creation, a statement in InfluxDBs query language can be specified:

from(bucket: "cloudsensor")
  |> range(start: v.timeRangeStart, stop:v.timeRangeStop)
  |> filter(fn: (r) =>
    r._measurement == "office"
  )

In the example above, the dashboard will display two graphs for temperature and humidity in the office.

Ansible

Our infrastructure from above consists of an InfluxDB and a Grafana service. Both have to be created and configured which involves:

• Starting the Services.
• Create volumes for them to store their persistent data.
• Configure user names and passwords.
• Configure Grafana to use InfluxDB as data source.

Ansible can help us here with its “Infrastructure as Code” approach:

• It provides a description language to document our service topology.
• The description language is executed by “Ansible Playbook” to automatically create the infrastructure for us.
• Once written, we can spawn as many Influx/Grafana services in our cloud provider as needed without much effort.

Inventory and Vault

The first ansible artifact which has to be created is an inventory:

influx-grafana:
  hosts:
    osiris:
      ansible_host: osiris
      ansible_user: belial
  vars:
    influx_user : christian
    influx_org : phobosys
    influx_bucket : cloudsensor
    grafana_user : admin

It contains a list of hosts and variables. In our case, we configure one host, a user name to access the host via ssh and Grafana/Influx credentials. The passwords are stored in an encrypted vault file called “secrets.enc”:

influx_pass : yourpass
grafana_pass : yetanotherpassword

The vault file is encrypted with a “ansible-vault”:

ansible-vault encrypt secrets.enc

Upon execution, ansible playbook asks for a password to encrypt the file.

Playbook

Besides the inventory, we also have to write a playbook which describes the target infrastructure:

• Configure docker on the target host.
• Create volumes for InfluxDB.
• Run an InfluxDB docker container and set its user credentials.
• Create volumes for Grafana.
• Run an InfluxDB docker container and set its user credentials.
• Create a virtual python environment on the target host.
• Execute the python script which creates a InfluxDB data source in Grafana.

First, docker is started on the target host:

- name: start docker
  become: yes
  service:
    name=docker
    state=started
    enabled=yes

The example above simply uses systemd to run and enable docker. “Become” instructs ansible to make use of sudo since we need admin privileges. Ansible asks the user for his sudo password when the command line parameter “–ask-become-pass” was specified.

The next example demonstrates ansibles docker capabilities:

- name: Create a grafana data volume
  community.docker.docker_volume:
    name: grafana-storage
- name: Create a grafana container container
  community.docker.docker_container:
    name: grafana
    image: grafana/grafana
    volumes:
      - grafana-storage:/var/lib/grafana
    published_ports:
      - 3000:3000

The example above begins with the creation of a docker volume named grafana-storage. Afterwards, it starts a container using the image grafana/grafana which is downloaded from dockerhub if not available. The container mounts the previously created volume and exposes port 3000.

Further Work

This article describes the automated creation of IoT infrastructure which stores, processes and visualizes data from temperature/humidity sensors. However, there are a few open tasks which will be tackled in further work:

• The database is exposed to the outside world without a middle ware in between.
• Not HTTPS is used.
• It is not possible to scale horizontal.
• Containers are started directly without an orchestration system like kubernetes, AWS ECS, etc.