The Forward Deployed Engineer, Chapter 17: Measuring Success with Outcome-Level Agreements

This is Part 17 of a series walking through my book The Forward Deployed Engineer. In the previous chapter, we made the inverse case. This one closes Part V with the measurement discipline the function lives or dies by: Outcome-Level Agreements.

The standard SaaS metrics — ARR, MRR, net dollar retention, NPS — fail for the FDE function for three related reasons, and the failures compound. They measure the wrong unit, treating revenue rather than outcome as the variable of interest. They lag the work that produced them by a year or more, so they tell you about deployments your team finished last summer when what you need to know is whether the deployment your team finished last week is healthy. And they treat the function as a cost center on the platform business’s P&L rather than as a leverage point on the platform itself, which produces the wrong incentives for everyone involved. The remedy I’ve seen actually work in mature FDE functions is the Outcome-Level Agreement.

An OLA is a written agreement between the FDE function and the customer that specifies, in operational terms, the outcome the deployment is meant to produce, how it will be measured, who is accountable for the measurement, and what the cadence of review will be. Each section is doing real work. The outcome has to be specified in language the customer’s own operations team would use, not in vendor terms. The measurement has to be defined precisely enough that there is no ambiguity about whether it’s been hit in a given period. The accountability has to be assigned to a named person on each side, with the authority to act on the measurement. And the review cadence has to be set at intervals tight enough to catch drift but loose enough not to consume the relationship in meetings. Most of the OLAs I’ve seen fail in production failed because at least one of these four was treated as a formality. They aren’t.

Once OLAs are in place, the function operates against an engagement-level dashboard per customer — outcome metrics, leading indicators, governance posture, and the platform-commit pipeline that connects the engagement to the platform team. The dashboard is not a slide deck the FDE team puts together quarterly. It is a product surface, owned and maintained the way any other product surface is, and visible to the customer on the same cadence as it’s visible internally. The discipline of building it this way is the difference between an OLA that holds for the life of the engagement and one that quietly degrades into vanity reporting somewhere around month nine.

Function-Level Metrics and the OLA as a Strategic Tool

Beyond the per-engagement dashboard, the function’s leadership tracks a small set of function-level metrics that aggregate across the customer portfolio. Platform commit rate, time-to-second-customer per pattern, productization velocity, sponsor retention, senior FDE attrition. Some of these are leading and some are lagging, and the chapter is precise about which are which and how to react to each. Sponsor retention is leading; once the sponsor pipeline thins, the renewals six quarters out are at risk. Senior attrition is lagging; by the time it shows in the data, the cycle is already three quarters old. The function’s leadership team needs both, but they need to know not to confuse them.

Building the OLA dashboard well requires a few specific roles to be clear. The FDE owns the construction — not the data engineer, not the customer’s analytics team, not a contractor. The FDE knows the workflow well enough to translate it into the right metrics, and the dashboard’s authority depends on that translation being credible to both sides. The framing for the customer matters too; the OLA dashboard has to be presented to the Chief Risk Officer or COO as a tool that helps them hold the vendor accountable, not as a tool the vendor is imposing on them. Done well, the framing turns a sometimes-defensive conversation about vendor performance into a collaborative one about deployment health.

The chapter closes with the OLA’s strategic role. In my experience, it is one of the most effective trust-building tools the FDE function has — an OLA that holds for two quarters typically converts into a multi-year contract, because the customer has now seen, in numbers they trust, that the deployment is producing the outcome it promised. And in the cases where the OLA fails fast — the outcome isn’t hit, the metric reveals the deployment isn’t producing value — the failure is itself useful information. It tells you, quickly, which customers to walk away from before the engagement consumes another year of senior FDE time. Both outcomes serve the function’s economic discipline. Tomorrow: the people who fill the function.