New to Honeycomb? Get your free account today!
This post was written by David Chang, Staff Software Engineer at Pinterest, and originally posted on the Pinterest engineering blog on Medium.
Develocity, formerly known as Gradle Enterprise, is a powerful tool that speeds up local and CI build time, helps troubleshoot your builds, and analyzes your data. At Pinterest, we have a dedicated team, Mobile Builds, and we ensure that developers can build fast and often. This enables developers to be more productive by getting faster feedback on their code.
We’ve been using the Develocity API to send data to Honeycomb for about a year. Honeycomb helps engineering teams deeply understand their own production systems through observability. If you’re already a Develocity customer, you might be wondering why you might want to build additional tooling to gather more data around your builds. In this blog post, we will cover three reasons you might want to send your Develocity builds to another provider for more processing, and we will discuss the high-level ideas on how to achieve that.
Get deeper insights into the build data
Honeycomb has provided us with a lot more insight than what is available in Develocity’s dashboards. Honeycomb’s query language is more robust and allows us to create more complex queries than is possible in Develocity.
Some examples include:
1. We’re able to group build times based on the user in order to find outliers.
2. We can compare build data between different machine types, allowing us to identify developers who are on Intel Macs and work with their manager to get them the ARM-based Macs.
3. We’re able to identify users who are not using the blessed JDK that is being excluded from our security software.
Identify issues faster with BubbleUp
BubbleUp is a Honeycomb feature that allows us to easily compare the differences between two groups of builds. We have used it in the past to identify slower builds for folks who had all their extra Android build variants and plugins enabled. We usually enable these automatically for CI to be able to build the release, and these incur an extra cost on the configuration and execution time for our builds.
Honeycomb is fast
The example below shows a 30-day query for the Develocity Performance tab and a similarly configured dashboard for Honeycomb.
In most cases, our Honeycomb queries finish in less than one second, which allows us to stay in the flow.
Develocity
Honeycomb
High-level implementation
Develocity offers an API to be able to access the data in your Develocity instance. At Pinterest, we have a CI job that runs every five minutes and queries two APIs /api/builds and /api/builds/{id}. We store a (/api/builds) to be able to use it in the /api/builds call. The first API call (/api/builds) gives us a list of the build scans along with some metadata, and /api/builds/{id} gives us more rich data for an individual build scan.
We then do some light processing of that data, and send it to Honeycomb using libhoney.
Since Develocity is the source of truth, it is likely that the data in Honeycomb will not be as complete, but it offers a good enough insight into our builds and we think it’s a fair enough tradeoff.
Conclusion
Integrating Develocity build data with Honeycomb has proven to be a beneficial step for our team at Pinterest. By leveraging Honeycomb’s robust querying capabilities and fast processing, we have been able to gain deeper, more actionable insights into our builds, and identify and resolve inefficiencies. While Develocity provides a solid foundation, the additional layer of analysis enabled by Honeycomb allows us to enhance our development processes even more. We encourage other Develocity users to consider this approach to maximize the potential of their build data.
