Update to Go SDK - Efficient Sampling

Here at Honeycomb we believe the best approach to handling high volume traffic is through sampling. As services scale, we want to continue to be able to inspect events with rich detail and context. In order to handle the traffic, we recommend sending representative samples of events, and scale sampling according to the value of the events being processed.

Our SDKs make it easy to do this kind of sampling in your code. When creating an event, you set a Sample Rate on the event and hand it off to libhoney. Our library takes care of only sending the right number of events and communicating the sampling to Honeycomb.

Here is an example using the Go SDK:

// create an event, add fields
ev := libhoney.NewEvent()
ev.AddField("duration_ms", 153.12)
ev.AddField("method", "get")
// set the Sample Rate
ev.SampleRate = 5
// send the event
ev.Send()

The code above will send 1 event for every 5 times it’s executed. When displaying your data, Honeycomb will take this sample rate into account and show you the correct totals for the graphs it displays. In the data mode views, you’ll see a column Sample Rate showing you that the row represents a number of other similar events.

Sometimes, especially in high volume scenarios, it can be expensive to generate the data you want to push into the event. In order to avoid spending the time to generate an event only to drop it immediately, you can do the sampling logic yourself. This allows you to only spend the energy to create the events that you will actually send.

// only create events 1/5 of the time
sr := 5
if rand.Intn(sr) == 0 {
  ev := libhoney.NewEvent()
  ev.Add("my_data": expensiveComputation())
  ev.SampleRate = sr
  ev.SendPresampled()
}

The SendPresampled() function does not do any sampling - it is used when the calling function has already taken care of the sampling logic. It is a little more complicated to use, because if the calling function does not sample correctly, the data represented in Honeycomb will be incorrect. However, the added flexibility will let you add instrumentation in expensive code paths while reducing the additional load on your service.

Have thoughts on this post? Let us know via Twitter @honeycombio.