Wish Google Cloud IoT Core Had an Emulator? We’ve got you covered
The Google Cloud IoT core is a fully managed service that connects, manages, and ingests data from globally dispersed devices. But it does not come with a built-in emulator, and we are not sure why. An emulator, as you probably know, offers a cost-effective alternative, reduces labor hours and has many more important advantages.
In short, its absence costs money.
When we first felt the pain…
While designing our backend architecture, one of the main design goals was handling the massive scale of data and devices. After evaluating all popular cloud providers and their IoT solutions, we decided to go with GCP and its IoT Core solution.
Our architecture consists of two main components: Agent software, which is installed on the IoT device, and AI driven Cloud, which sees the whole picture and makes smart decisions
Fast-forward to today, our backend contains dozens of microservices working alongside managed services such as Dataflow, Pub/Sub, and Cloud IoT Core.
The IoT core microservices – why an emulator is a must in this case
Microservices are great, but if not done correctly they can cause a slow and painful development cycle. A key feature in achieving rapid development cycle is CI/CD pipelines. At Firedome, the development methodology is to develop and test locally, while continuously delivering into a staging environment and manually deploying to the production environment.
Developing locally for GCP is usually done by using an emulator for the managed services.
Unfortunately, one of our most used managed services – Cloud IoT Core doesn’t come with an emulator. So we’ve rolled up our sleeves and created one.
In the following paragraphs, I will explain why and how to implement an emulator for managed services. Specifically, I am going to focus on Cloud IoT Core.
If you wonder why you should develop locally (emulator) as much as possible:
- Cost – Transmitting 50GB a month via IoT Core for testing purposes is very real for IoT companies. According to the GCP Pricing Calculator on the day of writing this blog post, running locally can save you about $3,000 a year.
- Velocity – Developing offline is much faster, especially if done in areas with bad internet connectivity, for instance, while commuting to work.
- Safety – If not done correctly, testing against the cloud may cause many issues due to data leaking from tests. Ranging from weird false bugs caused by two developers tests conflicting with each other, up to even destroying production data if done on the same GCP project.
On top of all these reasons, migrating to local development, even when writing an emulator yourself, is a relatively small task. The first step in converting a remote development part of your software into a local one is identifying the integration points between your software and the managed service.
The emulator we are sharing with you
(stay tuned, you will get a link soon :))
Our backend architecture relies on four main features of Core IoT:
- MQTT bridge capabilities
- Creating new registries
- Provisioning new devices
- Transmitting telemetry data into Pub/Sub
Now you can start coding. We decided to start with the MQTT bridge feature as the rest of the features are just incoming or outgoing REST APIs which should be straightforward to implement. After researching for popular MQTT brokers we chose mosca, an easy to use nodeJS MQTT server with a small fingerprint and friendly API.
Next up – implementing the transmission of telemetry data into Pub/Sub emulator. Integrating Pub/Sub’s REST API into mosca’s is pretty trivial:
The last two features require an HTTP server. We chose express, a very popular nodeJS HTTP server. The fun part about writing an emulator is that the implementation can be very simple.
Implementing the REST API parts can be as simple as storing device and registries information inside a data structure such as a map between registry ID to a collection of devices and defining the equivalent routes.
Here it is:
We are open-sourcing our version for IoT Core emulator. This is a basic version which we plan to expand in the near future with more IoT Core features. Contributions are very welcome :). If you are passionate about solving tough software engineering problems – come work with us!