Getting Started with Rails + Honeycomb

We hear you’ve got a Rails application you’d like to learn more about! Webapps like yours run the Internet, and we’re here to help you use Honeycomb to observe your Rails application. This guide will use the RailsBridge Bridge Troll application as an example (code here and site here), but you’re welcome to follow along in your own application instead.

Here are some example questions that this guide will aim to help you answer:

By the end of this guide, if you’ve followed along (and you should!), we should have:

Many thanks to the fine folks of RailsBridge for producing such a high-quality public Rails app for the community to learn from!

Preface: What has been staring us in the face already?

If you’re a Rails developer, you already know how wonderful Rails is for getting things to Just Work. And if you’ve spent any time looking at the default server output, you know how many timers are embedded in Rails itself— and how much useful information gets output to the (unstructured—what a shame!) server logs.

Example Rails log output

As you read through this guide, consider the following: What other sorts of data do you see in development all the time that might be helpful for debugging your application in production?

Set up honeycomb-rails

Using the honeycomb-rails gem it’s easy to configure your Rails app to send all that great metadata to Honeycomb. The Rails team built some instrumentation “hooks” into Rails itself, so that developers can be notified when certain events happen inside their application. honeycomb-rails takes advantage of some of these to capture events about what your Rails app is doing, and to forward them on to Honeycomb!

To get started, add the following to your app’s Gemfile:

gem 'honeycomb-rails'

That will configure Rails to send events to Honeycomb as your app processes HTTP requests and queries the database. We’ll be able to isolate specific controllers, actions, and formats; track successes and failures via HTTP status code; and drill down into how much time we’re spending in the database. Cool!

Note that libhoney, the library used to send events to Honeycomb, does all of its HTTP transmission on a background thread, so you can trust honeycomb-rails to have minimal impact on your application’s performance.

However, we still need to configure some credentials so we know which Honeycomb account and datasets to send events to. Add an initializer (we’ve named ours config/initializers/honeycomb.rb):

# config/initializers/honeycomb.rb

HoneycombRails.configure do |conf|
  conf.writekey = 'your honeycomb writekey here'
  conf.dataset = 'yourapp'
  conf.db_dataset = 'yourapp_db'
end

After clicking around a bit in our local Bridge Troll application to trigger some HTTP requests, we can start seeing data in Honeycomb: below, we’re exploring which controllers and actions are being requested the most and how long those requests are taking.

Count and P95(duration_ms) of controller/action pairs

Next: understand individual user behavior

For apps that use Devise for authentication, honeycomb-rails automatically adds information about the current user to the event sent for each request. This means we can analyze individual users’ Rails requests from inside Honeycomb! We can figure out which users are triggering the slowest requests, or even just see traffic patterns with a given email address.

For example, it’s trivial to filter down to a single user’s traffic:

Zoom into a single user's traffic trivially

Honeycomb Tip: Capturing user information in events is a classic example of a “high cardinality” field— meaning, there are tons of possible unique values for the field.

Including user IDs or email addresses can sometimes cause problems in other data systems— but Honeycomb makes it easy to filter your app’s requests by a given email address while also answering high-level questions (e.g. “Which endpoints are the slowest for this particular user?”).

In addition to any user information, honeycomb-rails also captures any flash messages that were set for the current response. They can be helpful if we ever have to debug an error or strange user interaction in the future. Breaking events down by flash_notice or flash_error can give you a unique insight into how your users have been experiencing your system.

Breaking events down by flash message can give you a unique insight into how your users experience your system

Honeycomb Tip: Adding fields like this won’t slow Honeycomb queries down, because Honeycomb’s query engine only pulls in the fields actively being queried over.

Recording application-specific information

The automatic instrumentation that honeycomb-rails provides out of the box has gotten us pretty far— we can answer questions around user behavior, performance anomalies, etc— but if we want to really understand the detailed interactions in our application, we’ll want the ability to capture things specific to individual controllers and actions.

Add useful fields for (most of the) whole controller

To record information in the scope of a particular request being processed, honeycomb-rails adds a honeycomb_metadata method to your controllers. It returns a hash, whose contents will be added to the event that we’re sending to Honeycomb. So recording information about a request is as simple as adding data to the hash.

As an example, we’ll instrument a classic Rails controller, Bridge Troll’s ChaptersController, which manages listing, displaying, updating, and (sometimes) destroying RailsBridge chapters.

Most actions on this controller work with a single chapter, so it’ll be helpful to track which chapter we’re operating on. The chapter is stored in the @chapter instance variable, set by the assign_chapter method. So we can extend assign_chapter to also record the chapter id in the honeycomb_metadata:

# app/controllers/chapters_controller.rb

def assign_chapter
  @chapter = Chapter.find(params[:id])

  # Add this line:
  honeycomb_metadata[:chapter_id] = @chapter.id
end

Because assign_chapter is set to run as a before_action hook for the actions that work with a single chapter, we can be sure that we’ll record the chapter for all those actions, without having to instrument each action individually.

Add useful fields for one important action on a controller

There are often specific things in specific actions that are worth tracking. In our ChaptersController#index, for example, it might be interesting to know how many chapters are being returned to the user.

# app/controllers/chapters_controller.rb
def index
  skip_authorization
  @chapters = Chapter.all.includes(:organization)

  # Add this line:
  honeycomb_metadata[:num_chapters] = @chapters.size
end

Honeycomb Tip: This is an example of a field that contributes to “sparse data”: sometimes not all fields exist on all payloads in a Honeycomb dataset, and that’s okay! Honeycomb handles these extra fields with grace, ignoring them if applicable (e.g. if you’re calculating the AVG(num_chapters), the query engine will ignore Honeycomb events without a num_chapters value set.).

Now, when everything is saved and we reload ChaptersController#index, the page loads and… our Honeycomb dataset now knows about num_events on requests for this controller and action!

As another example, we can add num_events to track the number of events returned for a given Chapter in ChaptersController#show:

# app/controllers/chapters_controller.rb
def show
  skip_authorization
  @chapter_events = (
    @chapter.events.includes(:organizers, :location).published_or_visible_to(current_user) + @chapter.external_events
  ).sort_by(&:ends_at)

  # Add this line:
  honeycomb_metadata[:num_events] = @chapter_events.size

  # ... Keep on keepin' on
end

Then (see the video below!) we can start with a graph of the COUNT and AVERAGE(num_events) of requests going to individual controller/actions, then adjust the query to break down by chapter_id instead (including only the controller/actions that contribute towards num_events). This lets us iterate on how we want to break down our app’s traffic, while always making sure we have access to the raw events.

Understanding database performance

To go beyond just controller activity, honeycomb-rails also sends events about SQL queries run via ActiveRecord events to describe how your application is interacting with the database. These events log the normalized SQL query, separating out the shape of the query (SELECT * FROM users WHERE users.id = ? from the parameters ["id", 1]).

A quick note on where we’re sending our data. Our events representing HTTP requests are currently going to a dataset in Honeycomb called “yourapp” (configured in config/initializers/honeycomb.rb above). The events representing SQL queries are instead going to a dataset called “yourapp_db”.

Honeycomb Tip: We recommend separating events into different Datasets when two events aren’t comparable in their frequency or their scope.

In the "yourapp" dataset, one inbound HTTP request maps to a Honeycomb event—this makes that dataset easy to reason about and analyze. Because a single Rails request may trigger many ActiveRecord queries, and because ActiveRecord queries have little in common with the controller or action that invoked them, it feels more natural to keep them in separate datasets.

Next Steps

With just a few lines of code you can get fine-grained visibility into the behavior of your Rails application, but this is just the starting point: the real power of Honeycomb comes when you instrument further. By adding data to honeycomb_metadata you can record detailed information about each kind of work your application performs.

Let’s take a look at the changes we made in this guide. Our full fork of the Bridge Troll app can be found on GitHub.

Some suggested next steps:

Bonus: If you’re not on Rails

If you’re looking for a Ruby-based solution but aren’t on Rails, we’ve still got you covered! This approach of mapping individual requests to Honeycomb events is also useful for folks to get started with analyzing their system.

Our Rack middleware (available via gem install rack-honeycomb) captures a number of the same things that honeycomb-rails captures: request path, request duration, and status code. It’s a little bit more involved to add custom metadata like we did in the last section, but it’s certainly still possible :)

# config/application.rb
require 'rack/honeycomb'

class Application < Rails::Application
  config.middleware.use Rack::Honeycomb::Middleware, writekey: "<YOUR WRITEKEY HERE>", dataset: "<YOUR DATASET NAME HERE>"
end