Today, AWS announced the AWS Lambda Runtime Logs API, a new way to easily send logs from AWS Lambda functions directly to your destination of choice.
AWS Lambda Extensions, announced in October, provide the ability to run code in parallel that is independent of your function’s lifecycle. We’ve created an extension that utilizes the Lambda Runtime Logs API to send your logs directly to Honeycomb.
Using our new extension, you can more reliably send Lambda events to Honeycomb while also decreasing any added overhead, latency, and cost.
AWS Lambda Runtime Logs API
AWS Lambda now automatically captures logs and allows you to stream them to custom locations using extensions. The stream contains the logs generated by your functions, the Lambda service, and Lamba extensions. Extensions can run logging tools to receive log streams directly from within the Lambda execution environment, and then send them directly to any destination. Logging extensions can also process, filter, and send those streams to any preferred destination.
Thanks to the new AWS Lambda Runtime Logs API, it’s now possible to send events from your Lambda functions directly to Honeycomb.
CloudWatch Workarounds, No Longer!
Serverless applications using AWS Lambda faced inherent challenges when sending data to Honeycomb. Lambda functions typically have very short runtimes. Any child processes are stopped when the function terminates. Honeycomb users sending Lambda event logs faced two options: ensure data integrity (but add latency) by blocking the termination of your function until outgoing queues are flushed, or optimize for performance but potentially lose some events.
Historically, the workaround for Honeycomb customers has been to send their Lambda log messages to CloudWatch. Forwarding those logs to CloudWatch Logs makes logging a non-blocking operation in Lambda. Our CloudWatch Logs integration listens to your CloudWatch Log Groups, parses them, and sends those events to Honeycomb.
This workaround strikes a balance between minimal overhead and data integrity. But it also has some drawbacks: notably, cost and little realized benefit from using the CloudWatch Logs service itself. The new Honeycomb extension now removes the need for you to implement this workaround.
How it works
Add the Honeycomb Lambda Logs extension to your Lambda function as a Layer. Then add your Honeycomb API Key and Dataset as environment variables in your Lambda function runtime environment.
That’s it.
As long as your Lambda functions emit structured data to STDOUT, you don’t need to worry about flushing queues or connecting to CloudWatch. If you were previously using CloudWatch, having completed those steps, you can then revoke your function’s permission to write to CloudWatch.
The new Honeycomb Lambda Logs extension decreases the overhead, latency, and cost of sending events to Honeycomb while increasing the reliability of that transmission. We’re excited to make this available because it should be a big win for anyone using AWS Lambda with Honeycomb.
Give it a shot and let us know what you think.
