SDK Spec

This document describes the requirements to which all Honeycomb client libraries must comply. The implementation and syntax for each of these requirements will differ by language.

Overview

A Honeycomb Client Library is intended to be used from within the code of an application. Its primary purpose is to make it easy to create events, add data to events, and send those events to Honeycomb. It must impact the performance of the application as little as possible, especially in failure scenarios. It may include extra functionality that makes it easier to add specific types of data to an event (for example, timers), but the focus should be on providing core functionality and allowing people to wrap the library for their convenience functions.

Throughout this guide the following vocabulary is used:

To be accepted as a client library, all criteria in the Minimum Necessary section must be met. The library must support:

The Fully Featured Library section details the complete spec for the most complete libraries out there. Only a few languages have libraries that meet everything outlined in the fully featured library spec.

Minimum necessary

Initialization

The library must have an initialization function to call. It should be an error to send events on an uninitialized library.

The initialization should start up any background threads necessary for transmission. The number of threads to use, the size of the pending and response queues, blocking, and so on should be able to be configured with the initialization.

You should be able to set any of the following during initialization:

We intentionally allow initialization without a writekey or dataset but will fail on event send as a result.

Events

You must be able to create an event, add multiple fields to it, and send the event.

You must be able to set the following attributes on an event:

If not set on the event, each of these attributes should default to what was set during library initialization.

Transmission

Sending events must be non-blocking by default. When triggering an event send, the event should be handed to a transmission component and the send() method should return immediately.

Sampling happens in the send() function - if the Sample Rate is an integer greater than 1, send() should only send events with a probability of 1/Sample Rate.

Transmission must use the parameters from the event being sent for the actual sending - it should not use or check any globally set attributes.

Transmission must send an event to the APIHost specified on the event using HTTP POST. The key/value pairs that comprised the data for the event must be JSON-encoded as the body of the POST.

The POST must be made to /1/events/<dataset_name>

The POST must include the following HTTP headers:

The POST should include the following HTTP headers:

Tests

Tests should cover

Documentation

The library must provide documentation in three ways:

License

The code must be released under the Apache License 2.0 as described here: http://choosealicense.com/licenses/apache-2.0/

Fully featured library

Initialization

The library must have an initialization function to call. It should be an error to send events on an uninitialized library.

The initialization should start up any background threads necessary for transmission. The number of threads to use, the size of the pending and response queues, blocking, and so on should be able to be configured with the initialization.

You must be able to set the following during initialization:

You may be able to initialize a null transmission, in which case no events will be sent (eg for testing or non-production environments).

Events

The minimum data required to send a valid event must be a (possibly inherited) single key/value pair. It is an error to attempt to send an event with no data, no APIHost, a missing writekey, or a missing dataset name.

The basic flow for an event is

Internally, an event must include at least the following attributes.

Internally, an event must include the following attributes, and allow each to be overridden on a per-event basis. All of these values should be initialized to those set in the global state or builder, and must be set before handing the event off to Transmission:

Internally, an event must include the following attributes, and allow each to be overridden on a per-event basis. These should not inherit values from the builder or global state.

If you add the same key to an event multiple times with different values, only the finally added value is kept and sent with the event.

Sending events must be non-blocking by default. More on this in the Transmission section below. Modifying an individual event does not need to be threadsafe.

Builders

Builders are optional.

A builder is an event factory - builders are used to create events with specific properties. A builder contains key/value pairs or dynamic fields and every event generated from that builder inherits those key/values as initial state. A builder must also allow writekey, dataset, sample rate, and APIHost to be set on the builder and events created from the builder will inherit those settings. The settings in a builder override those in the global state for events created from that builder.

Builders may be created by either instantiating a class or using a constructor. The constructor should be named something like new_builder()

The semantics of creating builders and adding data to builders should be the same as creating and adding data to events. A builder should have an additional method to create an event using the state of the builder, named something like new_event()

A dynamic field is a function which, when called, returns a key/value pair. The user of the library provides the function, and it is called once for each event that is created from the builder.

Global State

You should be able to add key/value pairs to the global state. You may be able to add dynamic fields to the global state.

If there are any fields set in the global state, every event should inherit those fields.

Responses

The library should have a method of providing the user with a record of Honeycomb’s response to each event sent. Getting data on the responses should be asynchronous in nature - the response should not be tied to sending the event.

Reading responses must not be a requirement to efficiently send data to Honeycomb. The user may be able to choose to block sending on making sure the response queue is consumed.

Any content the user provides in the metadata should be handed back to the user along with the response. This metadata is the method by which the user can tie a specific event to its response.

Every event must have a corresponding Response, even those that are sampled or dropped due to overflows.

A response object must have the following attributes:

A response may have the following attributes:

Transmission

Sending events must be non-blocking by default. When triggering an event send, the event should be handed to a transmission component and the send() method should return immediately.

Sampling happens in the send() function - if the Sample Rate is an integer greater than 1, send() should only send events with a probability of 1/Sample Rate.

Transmission should enqueue the event for eventual sending. It should have a configurable number of threads sending individual events. The default queue size should be 1,000 or 10,000. When the queue overflows, excess events should be dropped.

Transmission should use connection pooling.

Transmission should do batching internally, and if so, must have a method for triggering sending a batch based on either the entire batch filling or a timer triggering. The size of the batch and frequency of the timer must both be configurable. Sending may use the batching API endpoint for more efficient transmission. Suggested defaults are a batch size of 50 and a batch timer trigger of 100ms. When batching, the library must separate events into batches that all have API Host, writekey, and dataset in common, and send off batches as they fill.

Transmission may provide an option for eventually blocking sends. It should buffer a number of events before blocking. The size of the send queue must be configurable, and should default to 1,000 or 10,000. When blocking, an event send() should not return until the event is enqueued.

Transmission must use the parameters from the event being sent for the actual sending - it should not use or check any globally set attributes.

Transmission must send an event to the APIHost specified on the event using HTTP POST. The key/value pairs that comprised the data for the event must be JSON-encoded as the body of the POST.

The POST must be made to /1/events/<dataset_name> or, if using the batch endpoint, /1/batch/<dataset_name>

The POST must include the following HTTP headers:

The POST should include the following HTTP headers:

Sampling

Sampling refers to selectively dropping events. The Sample Rate is representative of the ratio of events to drop. A sample rate of 5 means that 1 out of every 5 events will be sent. A sample rate of 1 means every event will be sent. Sample rates less than 1 are invalid and should be set to 1

Sampling must take place when you call an event’s send() method.

Sampling must be probabilistic. In other words, if the sample rate is 2, it does not mean that every other event must be sent. It means that each event should have a probability of 50% that it gets sent.

If an event that is subject to sampling (i.e. a sample rate greater than 1), any event that is not dropped must have the sample rate attribute for that event set.

The send_presampled() method should set the sample rate according to the event’s attributes, but should not do any sampling. send_presampled() is used when the calling function samples traffic so the library should pass all events through unchanged.

Tests

Tests should cover

Documentation

The library must provide documentation in three ways:

License

The code must be released under the Apache License 2.0 as described here: http://choosealicense.com/licenses/apache-2.0/