Training Videos Observability Distributed Tracing
Intro to o11y Topic 13: Refining a Trace in Honeycomb
Description:
In this video, Developer Advocate Jess manipulates the trace view to get a better idea of what’s happening in the data. Sometimes a trace says a LOT- we want it to say just enough.
Note: This video assumes you have already connected your app to Honeycomb. Jessica Kerr is using a sample app called Sequence of Numbers. If you would like to download the app and follow along, you can do so using the process from Intro to o11y Topic 3. If you need help connecting your app to Honeycomb, see Intro to o11y Topic 4.
Transcript
Jessica Kerr [Developer Advocate|Honeycomb]:
Here I am at this long trace view and I have the index field showing. Wow, there’s a lot in here! There must be something else after this request, but how can I see it? When I hover over the span I can see three dots over on the left. Click that, and a menu appears. Hmm, let’s collapse spans with this name, maybe
Ah! Here’s the next span, the other part. Hmm, I can’t see the index anymore. To undo that, I go back to the three dots and expand spans with this name. Now let’s try this one that does show the index. Ack! Too much! Hmm, maybe if I “expand spans at this depth.” I do want the root span expanded That’s better! OK, I see that this call to fib of 9 sent a request for fib 8 and fib of 7. And inside here… And the request to fib of 8 sent a request for fib 7 and fib 6. And this other request for fib of 7 sent a request for indexes 6 and 5
Wait, here’s a request to fib of 6, and here’s another request to fib of 6. How many times does fib of 6 get called in this request? I can see here that it found index=6 inside the http.url It found six spans in this trace that include the text index=6 in some field name or value. And they’re highlighted… wait. 1 2 3 4, I see five. And this one isn’t 6, it’s 7. Oh wait, this is saying that somewhere in these 203 collapsed spans there’s something with index = 6. Let me expand those again. OK, now we see that these two spans are highlighted. This one is the client call out to index=6
And this one is the server receiving that call. So there’s two spans highlighted per call to index of 6 within this overall request for index of 9. Six spans found, two spans per call, there are three calls to index=6 in one call to index=9. What about 5? It found 10 spans with index=5, so five calls to index=5 in one call to index=9. This seems like a key question – how many duplicated internal calls are happening in each external call from the front end?