The Forward Deployed Engineer, Chapter 10: Governance, Risk, and Safe Deployment

This is Part 10 of a series walking through my book The Forward Deployed Engineer. In the previous chapter, we covered the outer loop. This one closes Part III with the discipline that decides whether any of the playbook can actually run in regulated environments: governance.

There is a temptation, when designing an enterprise AI deployment, to treat governance as a tax on the real work. The chapter argues the opposite. Outside the AI conversation, in industries that have actually built deployable software for two or three decades, governance is treated as the legitimacy layer of the deployment — the thing that lets the system exist in the customer’s environment at all. In the most regulated industries, the vendor whose deployment ships with an auditable trail wins the procurement battle the “more capable” vendor loses, and the FDE who reframes governance from constraint to product surface is the FDE who gets the second contract.

Before walking specific practices, it’s worth establishing a taxonomy of the risks that the governance discipline exists to contain. There are five, and each is owned by a different stakeholder in the customer organization. Model risk — the chance the model is wrong — sits with the head of data science or the chief risk officer. Data risk — the chance the inputs leak, drift, or get poisoned — sits with the chief information security officer. Workflow risk — the chance the redesigned workflow itself produces bad outcomes — sits with the operations leader. Vendor risk — the chance the model provider does something disruptive — sits with the procurement function. And governance risk — the chance the audit trail itself is incomplete — sits with the compliance team. Each remedy is different. Knowing which risk you’re actually addressing in any given conversation is the difference between sounding like a vendor and sounding like a peer.

The single most important governance artifact for any agentic deployment is the audit trail. Every decision the agent made, with the input that produced it, the prompt and tool calls that processed it, the reasoning surface it produced, and the output it acted on — all of it retrievable months later in a form a regulator can examine. This sounds simple. It is not. The most common failure mode is an audit trail that captures the output but not the reasoning, which produces a record that’s technically complete and operationally useless. The second most common is an audit trail whose retention policy is shorter than the regulatory window that applies to the data being processed, which produces deletion events that are themselves audit findings. Both failures are easy to miss in design and hard to fix in production.

Human-in-the-Loop, the Regulatory Landscape, and the Dashboard

The companion to the audit trail is the human-in-the-loop checkpoint — the explicit handoff from the agent to a human reviewer at defined points in the workflow. HITL design is a discipline of its own. Checkpoints that are too frequent become rubber-stamps; the reviewer learns that the agent is almost always right and stops paying close attention. Checkpoints that are too rare let errors compound past the point where review can catch them. The right cadence is workflow-specific and risk-tiered, and the chapter walks the design framework that mature deployments use to set it.

The 2026 regulatory landscape sits on top of all of this. The EU AI Act’s tiered classification, the US sectoral patchwork of agency-specific rules, and the specific compliance regimes that apply per industry — SR 11-7 in financial services, HIPAA and FDA pathways in healthcare, FedRAMP and DoD authorities in government, ISO 42001 cutting across all of them — together produce a compliance surface that the FDE has to be able to discuss at a peer level with the customer’s general counsel. The point is not to memorize the rules. The point is to know which ones apply before the kickoff meeting, and to design the deployment so that the audit trail and HITL checkpoints satisfy the relevant ones by construction.

The chapter closes on two operational artifacts. Vendor risk management is the discipline of treating your model provider as a vendor risk to your customer — because that’s what they are, even if neither of you would frame it that way during the sales cycle. And the governance dashboard is the real-time view of the deployment’s compliance posture that, in the most regulated deployments, becomes the surface the customer interacts with most. Once a Chief Risk Officer is checking the dashboard daily, the deployment has graduated from vendor relationship to embedded system. Tomorrow: how the company that invented all of this actually built it.