Feature flags are great and serve us in so many ways. However, we do not love long-lived feature flags. They lead to more complicated code, and when we inevitably default them to be true for all our users, they lead to unused sections of code. In other words, tech debt. How do we stay on top of this? Find out how Honeycomb’s trigger alerts proactively tell you to go ahead and clean up that feature flag tech debt!
The problem:
We wrote some new code in our Secure Tenancy service that is incompatible with the existing code in our web app. We still support the previous way that Secure Tenancy responds to the end point in question, and so we decided to feature flag the code that decides which end point to hit. The trouble is, we do not love long-lived feature flags. They lead to more complicated code, and when we inevitably default them to be true for all our users, they lead to unused sections of code. In other words, tech debt. Even worse, its tech debt that we inevitably forget to revisit. This thinking lead us to wonder, how can we help eliminate that tech debt, or at the very least, how do we remember to come back and revisit that tech debt? Do we create a ticket to check when all of our users have upgraded their instance of the service? Do we make a monthly calendar event to check and see if we can delete that lingering feature flag? How can we know when to do that work without the toil involved in checking every once in a while? Then we realized, we should just use Honeycomb to tell us.
Creating a derived column with regex
All of our integrations and customer-hosted services are instrumented to send us the user_agent string in the request header that tells us which version of a particular service or instrumentation our customers are using. As an event passes through an integration or service, it appends to the user_agent the string associated with what it passes through. For example, libhoney-py/1.9.0 beeline-python/2.9.2 via basenji/1.0.1. The part that we want is just the basenji/1.0.1 part, and really, just the version number of 1.0.1. Off I went to work on my regex.
It took me a few iterations, but I landed on something like this for my derived column. This pulled out just the version from that user_agent string. (You can read about using regex in derived columns here.)
REG_VALUE($request.header.user_agent, `.+basenji/([0-9/.a-z]+).*`)
However, that wasn’t going to be enough. I wanted to be able to say: Show me all of the teams whose version is LESS THAN 1.1. I know how to do that in our query builder, but you can’t use less than on a string field. So I needed to turn that version into a float. This lead me to the following derived column:
IF(
CONTAINS($request.header.user_agent, "basenji"),
FLOAT(
REG_VALUE($request.header.user_agent, `.+basenji/([0-9/.]{3}).*`),
)
)
This gave me a column I could filter where the value was less than 1.1
Creating a trigger to tell us when to delete the feature flag
From this query, I clicked on the ... icon in top right, and selected Make Trigger (Learn more about creating a trigger here)
I set up the trigger to alert the project channel when there was no longer any traffic less than 1.1. In the triggers UI, that would look like when the COUNT was <= 0
Conclusion
We can now sit back, relax and wait for the trigger to fire to tell us to go ahead and clean up that feature flag tech debt!
Many people think about using triggers or alerts only in dire cases where things have gone wrong. But, I’m here to remind you that you can use them to remind yourself about lots of different things about your system!
