Training Videos Observability Debugging
Intro to o11y Topic 10: Using BubbleUp to Find Distinguishing Factors in Honeycomb
Description:
In this video, Developer Advocate Jessica Kerr demonstrates how to use the BubbleUp tab to ask “what is different about the slow traces?”
Note: This video assumes you have already connected your app to Honeycomb. Jessica Kerr is using a sample app called Sequence of Numbers. If you would like to download the app and follow along, you can do so using the process from Intro to o11y Topic 3. If you need help connecting your app to Honeycomb, see Intro to o11y Topic 4.
Transcript
Jessica Kerr [Developer Advocate|Honeycomb]:
Let’s get to that heatmap of latency again. I click on the Honeycomb logo to reach the home screen, Then click on the latency heatmap. Then I pick the range where the action is and click the magnifying glass to zoom in. I see in this heatmap that some of the requests are way slower than others. But why? Is there some systematic reason, or is it random? It doesn’t look random to me. Honeycomb has a magical tool to answer just this question. To use it, I click BubbleUp.
The heatmap looks the same, but it’s going to respond differently when I click on it. Back on the results tab, I dragged to zoom in. In BubbleUp my goal is to draw a box around the slow requests. I click on the heatmap and then drag to form this yellow box around the ones that look suspicious. Honeycomb looks at the events inside the yellow box, and at the events on the graph but outside the box. It does a statistical analysis on every field in these events and says, what is different?
Every field in the wide events we send to Honeycomb is a potential clue. BubbleUp searches them for answers. The results come up in these little bar graphs. Blue bars represent the events outside the box, and yellow bars represent events inside the box I drew.
It compares EVERY field; I can click on this arrow to see even more. But the most interesting ones are right up top, because those are the ones that are most different. I can see that duration has different values for the blue vs yellow, uh yeah that’s the point, they’re slower. This is the part where my own knowledge of what the fields might mean combines with what Honeycomb can tell me.
How about this one, http.url? I know that’s likely to be a meaningful field, and BubbleUp is telling me that it is different between a typical request in the yellow box vs outside. I point at that field, and it pops up that it’s populated in 100% of both events, OK.
Mouse over the bars to see the different values. It looks like… OK, this looks like it might matter. I want to see more. Click anywhere in the field’s graph to get some options for further investigation.
Most of these options will modify my query, so I can send a new query to Honeycomb. There are multiple values of URL here that are interesting, so I choose “group by field.” What happened? Nothing. Well, technically, the query up here is different. But you won’t see the difference until you click on the results tab.