Enjoy this Business Impact Brief from 451 Research.

The 451 Take

With the understanding that performance and functionality are crucial to business success, many companies are taking a new approach to analyzing the performance and usage of their applications. While collecting the right operational data about an application is key, making sense of that data by defining important goals and measuring progress against those goals is an increasingly popular approach. Best practices for this concept of service level objectives (SLOs) are emerging, with clear benefits for both engineering and business teams. Whether business-critical software is targeted at internal or external users, SLOs can help teams work together to drive the kind of application performance that users demand.

Most Software Users Will Switch Based on Performance

Source: 451 Research’s Voice of the Connected User Landscape: Consumer Population Representative Survey, Q1 2019
Q: If an online app/service you use performs poorly, how likely are you to switch to a different brand/provider as a result? (Excludes Don’t Know; n=1,309)

In our 451 Research Voice of the Connected User Landscape survey, 79% of consumers said they are very or somewhat likely to switch brands or providers due to poor performance. When revenue-generating, customer-facing apps and software that runs important internal processes perform poorly, they can directly impact the bottom line.

Business Impact

SLOs ARE BECOMING VALUABLE IN ACHIEVING VISIBILITY.

Defining and proactively tracking SLOs, a concept popularized by Google as described in O’Reilly’s Site Reliability Engineering book, is emerging as a valuable way to achieve the kind of visibility that enables engineering, operations, and business teams to make smart decisions that improve the user experience and ultimately avoid losing customers to competitors. Using SLOs requires some up-front work both to define effective objectives and to put in place the right tools to track SLOs. We find that both the process and the output have important impacts on organizations that take the time to embrace SLOs, which measure important objectives; one common type of SLO centers on latency, such as: 99% of requests return in under 100ms. Setting and tracking this SLO requires teams to measure each user’s experience over time.

AN SLO THAT ISN’T REALISTIC OR BASED ON IMPORTANT OUTCOMES WON’T BE BENEFICIAL.

We recommend starting with just a few SLOs focused on critical aspects of a service rather than potentially overwhelming a team with too many goals. Some teams may benefit from grouping SLOs, such as several latency related SLOs pegged to individual requests, in order to measure the broader performance of a service.

ORGANIZATIONS CAN USE ERROR BUDGETS TO MAKE IMPORTANT DECISIONS.

Forward-thinking teams that embrace SLOs are turning to modern tools that can compute and display an important related metric: the error budget, or how much time in a month that a system can fall below a specified threshold in order to satisfy the SLO target. Error budgets can help organizations make important decisions, such as allocating efficient human and technology resources. For instance, if an engineering team sees that it is exhausting its monthly error budget early each month, it can allocate engineer time to improve the performance of the service rather than work on new projects.

ADOPTING SLOS CAN IMPROVE COLLABORATION.

While the previous points are clear wins that most engineering and operations teams aspire to when adopting SLOs, there are additional important benefits. For example, setting SLOs often requires insight from business stakeholders, such as product managers that have an understanding of customer requirements. Collaboration between these teams can help build a strong relationship, enabling better alignment and achievement of shared goals. It not only helps engineers use their time more efficiently, but it also helps business teams understand what is involved in reaching and maintaining stated objectives. In addition, collaboration often introduces business teams to the insight available via the tools that engineering teams use to collect and analyze system-level operational data. With access to this visibility into application performance, business teams can make smarter decisions about the types of features that are popular with users.

Looking Ahead

Most organizations that are embracing the development and tracking of SLOs have been experimenting with the process, and we expect best practices to emerge in the short term. Businesses will need to define the SLOs that serve their unique users and that can evolve over time, but as organizations share their successes, the wider market will benefit from their experience. So too will organizations benefit from the ongoing development of tools that are flexible and support the array of data collection and analysis that enables tracking progress against an SLO.

As more organizations improve application and business performance through the use of SLOs, we anticipate that some business teams will begin driving this practice, rather than engineering teams introducing the concept. These business teams will spur a further evolution of SLOs that promise to highlight the potential of SLOs to support wider business objectives.