ESPHome – building composable and reusable Arduino projects

The Arduino Ecosystem as Foundation

I have experimented with various electronic projects and have been following the Arduino project for a long time. Actively I have programmed microcontrollers since the ESP32 chips came out, because they are much more powerful than the early Arduinos - that means they have Wi-Fi and Bluetooth combined with a speedy processor and other perks on board. 

If you look at the software landscape, Arduino IDE is the best known platform for this. The Arduino project has a uniform basis interface, for example for Wi-Fi and such things. That means that many common foundational APIs were defined by the Arduino project - which is a great progress compared to embedded programming before.

However, as Arduino is targeted to less experienced programmers, there are some aspects that I find more difficult as an experienced software developer. For example, there is only global dependency management: That means you install a dependency globally on your system in the latest version. You have to click onto lots of buttons in the IDE to install packages, and update them - making it quite challenging to get reproducible builds which work on other machines and/or way later than when they were initially created.

Platform.io for reproducible builds

To solve the above issues (and some more), another Open Source project named platform.io was created. It is a generic framework for embedded development, which excellent support for the Arduino ecosystem and chips.

This solves many of the above Arduino problems, for example there is declarative and version-controllable dependency management included, and a config file is used to define various platform specifics. Furthermore, the build process is very customizable via Python scripts.

This left me with one last problem to solve:  Typically, every hardware vendor (e.g. an M5Stack ESP32 board, an e-paper display with ESP32, ...) typically has its own SDK. Each of these comes with its own conventions and functionalities, making code built on this platform less portable.

I always wondered how to build something more portable, while not rebuilding the basics again and again.

ESPHome - A Library of Composable Components

Last week, by chance, I got a tip from a friend of mine about the ESPHome project, being created by the same folks who build Home Assistant. It works like this:

You create an Application Specification inside a YAML file. There, you compose together different components like sensors, outputs, displays, GPIO Pins, ...

Then, this application specification generates a platform.io project, and generates wiring C++ code (inside a file called main.cpp) which sets up the components and regularily runs them. Then, the platform.io project is built and installed onto the device.

At first I thought this was rather restrictive (because the project targets non-programmers explicitly, so the customizability is a little more hidden). However, it was very easy to customize all parts via custom C++ code (via a mechanism they call "lambdas" or "templates"). This way, we can easily write the code the way we want to.

What is the benefit of all this? Well, esphome brings lots of components out of the box which are guaranteed to fit together, AND we can extend it easily with custom C++ code. Some examples:

• mDNS, basic TCP networking
• auto-registration with home assistant, as well as bidirectional data flow from/to home assistant (based on the above)
• lots of existing sensors (for environment, light, presence, distance, temperature, ......)
• lots of output devices (e.g. LCD panels, ePaper displays, single LEDs)
• a unified logging and debugging system
• over the air updates and fleet management - this is HUGE in case you have many ESP32 devices out in the field.

Finally, the high level ecosystem I was dreaming of when programming ESP32 controllers 😍😍