Gamers Won’t Wait: Dead By Daylight Gets Some Sweet Attention
5 minute read
About
Behaviour Interactive (BHVR) is one of the largest independent video game developers in North America. They have developed over 250 titles and sold over 60 million copies on all major platforms.
Environment
- Node.js stack on AWS
- Multiple external platform services
Goals
BHVR’s engineering team had been using a classic APM approach for some time to troubleshoot latency issues in their flagship multiplayer video game, but the tool kept coming up short. Dead by Daylight is accessible on all major gaming platforms and it was proving impossible for the engineering and operations team to identify the source of a service slowdown—was it in the caching, the database, or somewhere in one of the various external calls?
This was starting to negatively impact BHVR’s customers who expect to play as quickly and smoothly as possible. Every time a user logs in via a given gaming platform, the BHVR service has to call that platform to get the player info for that user—and the stack was getting hung up, hitting a hardware limit that prevented the OS from opening any more sockets—preventing the user from playing.
There was no real way to find possible culprits with our classic APM. We had to know what we needed to find before we could find it—a dead end.
What They Needed
- A powerful and intuitive interface for debugging and troubleshooting problems in real time
- The ability to drill down into individual requests and identify the source of latency across multiple services and providers
Honeycomb @ BHVR
After using a traditional APM for some time but continuing to face challenges, BHVR decided to try Honeycomb. They started with a trial during which they successfully solved production issues in a fraction of the time it took before. They now rely entirely on Honeycomb to debug and solve production issues.
It took 5 minutes for my engineers to convince me that Honeycomb solved the problems that our previous APM could not—and I’m not easy to convince.
We were able to figure out that the issue was DynamoDB being throttled, and address that.
Using Honeycomb’s BubbleUp feature, which accelerates troubleshooting with heatmaps that quickly display any outliers compared to a baseline, the engineers at BHVR solved a showstopping problem while shipping a recent major release within a few minutes of discovering the issue.
Our producers were delighted to see issues solved so quickly. Developers can get quite irritated when facing a critical issue that takes more than a few minutes to resolve. Honeycomb saves us from that.
Not only is Behaviour solving new problems as they arise, they’re now able to go back and dig into problems they had given up on ever solving previously—and increasing the performance and resiliency of their code every step of the way.
I believe any company running a complex software stack would benefit from using Honeycomb—not just gaming companies.