Quindío Inteligente: serverless IoT ingestion

Quindío Inteligente was a project of the Grupo de Innovación EAM, in Armenia, carried out with EDEQ. Its technical goal was to receive data from IoT devices and show it in real time monitoring.

The devices send messages continuously and in high volume. The system had to absorb them without losing the pace of real time monitoring, and do it without paying for infrastructure that sits idle.

I was the cloud and IoT solutions architect. I designed the serverless architecture of the project with AWS Lambda and Docker and built the Python backend that ingests the messages over MQTT.

1. Serverless architecture with AWS Lambda and Docker

   Compute is paid for when it is used and follows the message volume, with no servers of my own to manage. With Docker, the same code runs the same way in development and in the cloud, which makes it easier to test the ingestion before deploying it.

2. MQTT with Mosquitto as the entry point for the devices

   MQTT is a lightweight protocol designed for IoT and Mosquitto is a mature broker. They fit devices that publish small, frequent messages, and they let the backend receive what gets published instead of polling the devices one by one.

3. Ephemeral containers and on demand services

   Whatever is not processing messages does not stay on, and that reduced cloud costs. An ephemeral container is created to do work and disappears when it finishes, so cost follows the work done and not the time a service stays up.

Three demands that pull in different directions had to be combined: high message volume, real time response and a controlled cloud bill. More volume usually asks for more capacity kept on, more speed usually asks for reserved resources and lower cost asks for the opposite. Every architecture decision balanced capacity, delay and cost.

I delivered a serverless architecture and an IoT ingestion backend that feeds the real time monitoring of Quindío Inteligente. Ephemeral containers and on demand services reduced the cloud costs of the project.

• AWS Lambda
• Docker
• Python
• MQTT
• Mosquitto