Using Nats.io as primary data-storage
Since recently, at Sandstorm we have a dashboard showing the quality of our projects. So far, it shows the latest Google Lighthouse results. We execute Google Lighthouse during our continuous integration and continuous deployment pipeline, and we are planning to include other metrics from other tools.
The nice thing about this dashboard, the application itself, is stateless. It has list of project or metrics and no database. So but how does it work?
Recently we introduced a Nats.io high availability cluster into our infrastructure. Oversimplified, Nats.io is a message broker. Applications can send message to or receive messages from it and do not need to communicate directly anymore.
Communication over Nats.io is easier to configure than individual connections, it's easier to work with and easier to extend. Especially since you can add add applications to the listening side and to the writing side without touching the existing ones. Beside other things, we use Nats.io to send metrics from our CI/CD jobs to the dashboard.
Nats.io allows you to record and replay messages. They call it JetStreams. So in our case for the dashboard, the metrics are messages on the JetStream. Those messages are replayed and read to render the dashboard. In the application we use the JetStream just like a database. It's not (yet) live updating. Another nice thing about JetStream: it allows you to configure a data retention policy. In our case we store five versions of the same metric for at most 30 days.
Since adding new application to the consumer side of a Nats.io JetStream is very easy, local development is very easy as well. You start a local version of the app, you read the events from the very same JetStream and can work on life data locally.
The dashboard itself becomes extremely simple. No user management, no receiving and storing data, only fetching events. It consists of 180 lines of TypeScript and 90 lines of markup.
Talking about code, the implementation is based in Deno and the according JetStream client.
As always you can use the code snippet as you like. It is licensed as MIT. If you have comments or feedback, feel free to contact us and thanks for reading.