A while ago, we added Metrics to our observability platform so teams could easily see system information right next to their application observability data—no tool or team switching required.
So how can teams get the most out of metrics in an observability platform? We’re glad you asked! We had this conversation with experts at Heroku. They’ve successfully blended metrics and observability and understand what is most helpful to know. Here are three strategies to maximize the benefits of Honeycomb Metrics.
Ask all the questions
It’s no secret that at Honeycomb we like to ask all. the. questions. And it turns out that’s good advice even when it comes to system metrics. Knowing that builds are failing isn’t going to be sufficient to understand why, and what needs to be fixed. Instead, focus on making sure the metrics are asking the key questions. And that the team is also comfortable with the questions—and the uncertainty.
It’s important to understand that, while wide events are amazing diagnostic tools, they aren’t one size fits all. Not every metric data point can fit comfortably into the “wide event hole.” Memory usage, CPU usage, and queue size are examples of metrics that don’t play nicely with a wide event.
Organizations used to poking and prodding wide events in order to find and fix instances may have a bit of a process adjustment when they treat metrics in the same fashion. Especially for companies with older systems in place; older systems tend to come with quite a few custom metrics that teams are incredibly familiar with, and we all know that change is hard. Time, team champions/resources, and practice can make the process easier and perhaps reduce the need for custom metrics moving forward.
In the meantime, keep the analysis loops going and ask all the questions.
Start with OTel, then customize
OpenTelemetry, or OTel, is the Cloud Native Computing Foundation’s open source standard for transmitting telemetry data to backend observability platforms. OTel is not an observability tool itself, but it provides a standardized way for organizations to work with a variety of platforms and providers in 11 different languages currently. Teams can use the OTel native protocol, OTLP, to transmit telemetry data, or take advantage of the OTel Collector to bridge the gaps.
The real genius of OTel, though, is that even teams who have metrics data awash in proprietary formulas can still join in the telemetry fun. The Heroku team crafted a custom receiver to surround its existing metrics and then convert them into OTel, which can then be sent to observability platforms of choice. It’s just a week or so worth of effort to have portable metrics, thanks to the stable OTel platform.
Map metrics to events
Teams comfortable working with events but still struggling with the idea of incorporating metrics will want to know about timestamps, which roll all data points into a single event provided they are in the same OTLP request and have other shared values. This effectively turns metrics into wide events and the data is presented in columns.
Timestamps have several benefits including drastically reduced event throughput (because all those data points are going to be processed as a single event) and the ability to use drive columns, which are both useful and highly digestible.
And, bonus, the volume reductions experienced with timestamps will also make it easier to work with OTel. This is particularly true for teams trying to add customization.
Watch the entire webinar: How Heroku Uses Metrics Within Honeycomb
Excited to learn more about Honeycomb Metrics?
Take a deep dive into our documentation
Learn how Honeycomb Metrics can bridge the gap between monitoring and observability
