A little bit of context
Let’s say you have a fleet of IoT devices and you constantly add features or improve them. How can you make sure you don’t break them in production? Here is the story of how we solve that problem for Sailsense Analytics. Sailsense monitors a diverse fleet of yachts to provide continuous safety alerts and telemetry for the yacht owners. The challenge stems from the high degree of customization each yacht owner requests - creating a wide range of configurations - combined with the firmware updates occurring several times per month to follow the company roadmap.
Naturally, it's not that straightforward. These IoT devices often run in remote environments with limited power, CPU, and RAM, making the highly complex issue of firmware management even more challenging.
The solution
Together with the embedded development team at Sailsense, we identified key metrics to assess the quality of new firmware versions, such as the performance of the GPS filtering system, data loss, and connectivity instability.
We leveraged Clickhouse’s power and Pandas speed to make a batched anomaly detector, generating a stream of anomaly events, per yacht, focusing on key signals such as GPS teleportation, discontinuities, network disconnections, etc. The anomaly classificators minimizes noise by masking less critical anomalies, ensuring the focus remains on significant issues. Finally, the system tags each anomaly with information such as the firmware versions and yacht attributes to group on, depending on the analysis later needed.
Data visualization is handled using Grafana, optimized for time-series analysis to track trends and performance over time. The solution was developed iteratively in collaboration with Sailsense’s lead developers to ensure it met the unique needs of the yacht fleet.
Benefits for Sailsense
This solution gives release managers a quick, visual way to evaluate firmware quality across multiple dimensions. It also provides management with digestible reports, allowing them to understand the impact of new firmware releases on yacht operations and make informed decisions.
The anomaly detection system provides Sailsense with real-time visibility into firmware performance, allowing them to quickly identify and address issues before they impact paying customers. This early detection leads to significant cost savings by reducing manual testing and ensuring a smooth release cycle. As a result, customer satisfaction is improved by ensuring higher reliability and minimizing hardware downtime.
Moreover, the system is designed for scalability, allowing it to expand alongside Sailsense’s growing fleet while maintaining high-quality control. Its intuitive reporting system offers actionable insights to both release managers and executives, enabling quicker decision-making and more streamlined operations.