Unlocking Smiles: HappyCo's Observability Success 

Challenge

With a diverse range of applications, HappyCo sought to advance their system investigations with a modern observability solution while embarking on an application refactor project.

Since its start in 2011, HappyCo has experienced rapid growth through both organic expansion and strategic acquisitions. As a result, the company has a diverse range of applications for customers to smile about. But, without an observability solution in place, maintaining these applications and debugging issues was becoming a greater challenge. 

Developers had to sift through logs without clear visibility into errors or performance metrics. "Investigating our systems was frustrating. It required at least four or five different hops across various logging infrastructures and database queries. Obtaining the necessary insights could take hours, and for more complex issues, even multiple days," said Josh Clayton, Senior Director of Engineering at HappyCo. Josh added, “This was hindering our ability to address our customers' needs in a timely manner.” 

As the engineering team embarked on a strategic project to refactor one of their monolithic applications, they reached a turning point. They realized they needed a comprehensive observability solution to help them successfully manage this significant undertaking, focusing on identifying and solving performance and latency issues. This new focus on observability was essential to ensure efficient and effective debugging, and to identify and solve performance/latency issues crucial for overall project success.

Solution

In search of a modern observability solution to empower its development team and support day-to-day operations, HappyCo initiated an evaluation of multiple solutions, including Datadog and Honeycomb. Honeycomb swiftly emerged as the standout choice.

"It’s no small project to refactor software, so choosing the right observability solution was crucial. We were impressed with the usability and the insightful information we gained from Honeycomb, enabling us to access meaningful intelligence and make informed decisions as we navigated through the refactor project,” Josh shared.

Several compelling factors influenced HappyCo’s decision to select Honeycomb, including:

• Cost efficiency
  Offering robust capabilities at great value, Honeycomb provided a transparent pricing structure with predictable costs, with no penalty to instrument rich high-dimensionality and high-cardinality data.
• OpenTelemetry
  "Honeycomb's native support for OpenTelemetry was compelling to us. Implementing OTel standards was a huge motivator for selecting Honeycomb as it aligned well with our involvement in open source communities."
• Usability
  "Choosing Honeycomb was a no-brainer. The usability and value Honeycomb offers are incredible, particularly in terms of the insights we can glean and the decisions we can make. And these decisions aren't trivial; at the time, they involved allocating teams to focus on app re-architecture, a significant undertaking that we didn’t take lightly."

Results

Onboarding observability best practices

Since their first interactions to their ongoing engagement with Honeycomb, HappyCo has experienced top-quality support. Josh highlighted, "Since our initial interactions with Audrey Herndon in customer success and our account manager, Amanda Wagner, our experience with Honeycomb has been incredibly supportive. The dedicated Slack channel and access to engineers has facilitated positive collaboration with Honeycomb, proving to be instrumental in our success. Attending Honeycomb's Observability Day with one of my engineering managers was pivotal, allowing us to meet the team and gain valuable insights into observability 2.0. Being able to ask questions and receive thoughtful responses has made our experience with Honeycomb incredibly valuable."

Equipped with a breadth of hands-on resources, HappyCo established an initial internal team of Honeycomb champions to build out instrumentation, establish the range of use cases, and develop a best-practice observability approach. This methodical approach ensured the champions were effectively leveraging the tool and could then train the whole team of 45 engineers. 

"Once our internal champions established our observability best practices with Honeycomb, we ramped up the other team members and started to advance everyone's maturity with the tool. Our engineering team is spread across the globe, so we come together once a month to speak on various topics. We had a full-team session about Honeycomb, and also created a Slack channel where we can chat async about various telemetry topics surrounding Honeycomb,” said Josh.

Enhancing performance with precision 

With Honeycomb in place, HappyCo gained the essential visibility they needed to navigate their monolith system refactoring project. "With Honeycomb’s supported instrumentation, we could clearly identify where we had performance issues, like areas with high traffic and slow response times. Those insights drove our project management efforts to systematically prioritize where we focused our efforts,” Josh shared. With Honeycomb's observability platform as the team’s cornerstone, they successfully completed the 18-month project and significantly improved the app's performance.

By leveraging Honeycomb's observability platform, HappyCo elevated their engineering team's efficiency and confidence to new heights. The platform provided them with real-time data insights and the ability to validate changes before deployment. This capability not only streamlined their development process but also ensured that issues were resolved quickly and effectively. 

As Josh explained, "One of the things that we absolutely love about Honeycomb is that we can instrument it locally and push the data up to see the before and after. It's still mind-boggling to me how fast the feedback loop is—it's instantaneous. So, we can validate with production-like data locally and have a high level of confidence that we did resolve the issue.”

Solving mysteries like expert detectives 

Now that the team has successfully tackled their bigger initiatives with Honeycomb, they are advancing their engineering practices by becoming proactive. During their development cycles, they take time to run queries and assess system performance. This proactive approach led them to uncover a curious mystery; the engineers discovered unusual traffic patterns with 246 hits per second on a specific page, which initially appeared to be a DDoS attack on the system.

Josh explained, "We added attribute tracing from Honeycomb to track which user was making which request, and that brought complete clarity for us to solve the mystery. Our virtual maintenance team had a browser extension refreshing their work order page every two seconds and the system response times were creeping up to five seconds. Once we understood the scenario and the maintenance team’s needs, we could address the latency.” 

Honeycomb empowering the engineers to get curious and have full access to explore HappyCo’s software products has enabled the team to solve mysteries and make meaningful usability improvements for the business and customers. Celebrating the victory, Josh underscored, "We felt like we were unlocking mysteries—like geniuses. It was this 'aha' moment of total clarity. Being able to figure this out with Honeycomb was awesome.”

Enjoying a “mind meld” usability experience

With Honeycomb, HappyCo’s engineering team has advanced their development practices and increased their efficiency with an intuitive observability solution that’s simple for everyone to use. Josh explained, “With automatic instrumentation, there's minimal effort needed to send data to Honeycomb, and querying it is instantaneous, giving me answers with virtually no effort. It's so easy to get insights to questions that I wouldn't have even thought to ask before using the product.”

Josh added, “Honeycomb’s speed and intuitive design means I don’t spend time figuring out how to get what I need. Honeycomb just gets out of my way, letting me concentrate on solving problems without any obstacles or delays. The system is fast and responsive, presenting data visually so I can immediately see when something is going sideways or not working. And investigations that used to take us hours or days can now be done in minutes."

Interested in learning more? Book a consultation with our sales team.

At a glance

About

Founded in 2011, HappyCo provides software and services for property management companies (PMCs) of all sizes that are changing the way millions of rental units are being managed across North America and beyond. The company’s portfolio of lifecycle support tools empower everyone in multifamily to improve their operations and maximize returns, which means they’re building better experiences for their residents. 

Industry

Property Management Software

Products

Honeycomb platform

Use cases

Platform engineering and debugging

Application performance monitoring

Results

• Boosted application performance, delighting customers with faster software responsiveness
• Accelerated debugging from days to minutes
• Streamlined development processes, increasing team efficiency
• Fostered a high level of confidence to push out code more readily, trusting it will work as expected
• Achieved significant cost efficiency through advanced observability practices

"Honeycomb is an integral part of our company’s success. When we think about providing stable, reliable service for our customers—because they rely on us to operate their businesses—our confidence in our capabilities is significantly boosted with Honeycomb. It helps surface the information we need to act on, ensuring we deliver that high level of quality for our customers."

Josh Clayton, Senior Director of Engineering at HappyCo