Your AI Pilot Didn’t Fail, Your Operating Model Did
By Kalyxi ·
AI pilots fail when detached from systems and SLAs. Embed into workflows with controls, then pilot in the live path with guardrails.
Most AI pilots stall because they run outside the live path, not because the models are weak. If we do not build for the systems of record, SLAs, and controls we already run, the pilot will look good in a demo and die in change advisory.
Why did the pilot stall when the demo crushed?
It stalled because the pilot lived in a sandbox that never touched systems of record, did not meet SLAs, and had no owner after the demo. The model answered questions in a chat window. It never wrote back to ERP, never logged to your SIEM, and never passed access reviews. When the time came to move from show to serve, there was nothing to attach to the real process. Operations said no. Risk said not yet. The sponsor moved on.
We have seen the same story across support, finance, and supply chain. The model performs fine on curated data. The gap is the operating model. No integration, no runbooks, no observability, no change plan, no go.
What operating model should we design for?
Design for production from day one: a service that can sit inside an existing workflow, read and write to systems of record under role-based access, meet response time and accuracy SLAs, emit structured logs and metrics, and be owned by a named team with a calendar and a pager.
That means we define the service interface, the identity model, the data contracts, the observability signals, the escalation paths, and the upgrade path before we train or prompt anything. We assume the pilot will serve real users. We plan for rollback on the first commit. We treat the model as one component in a controlled system, not the system itself.
What goes wrong when we sandbox AI away from systems of record?
The model learns tasks, but the workflow that creates value remains unchanged. The handoffs do not move, the clicks do not disappear, and audit gaps appear. Reconciliation gets harder because we created a second source of truth. SLA risk goes up because we added a parallel lane without clear routing.
When a pilot does not integrate with ERP or CRM, four things happen:
- No write-back means humans still key in the output, so time savings vanish.
- No identity integration means access reviews fail and we bottleneck on service accounts.
- No observability means we cannot prove accuracy or latency, so risk blocks scale.
- No change story means frontline teams see extra steps, not fewer, so adoption drops.
How do we run a live-path pilot without blowing up SLAs?
We route a slice of the real workload through the service under explicit guardrails. We scope volume, case types, and time windows. We set strict thresholds for auto-apply versus human-in-the-loop. We prepare a fast rollback. We start small, we never start fake.
A workable pattern looks like this:
- Case selection: 5 to 10 percent of volume in a single region or queue, low-risk categories first.
- Controls: Human review required for changes above defined thresholds, for example credits over a set dollar amount or orders to flagged accounts.
- Dual run: For the first two weeks, the service drafts actions but a human submits them. In week three, the service submits actions under thresholds, humans review exceptions.
- Telemetry: Every decision logged with input, features, prompt version, model version, and outcome. Latency SLOs set, for example 1.5 seconds p95 for retrieval, 4 seconds p95 end-to-end.
- Rollback: One switch in the router returns traffic to the baseline path in under five minutes.
What does good governance and controls look like for production AI?
It looks like the controls you already run for production services, applied specifically to model behavior. We bind identity to entitlements. We narrow data scope to the minimum fields needed. We log prompts, responses, and actions. We put humans in the loop where risk is above tolerance. We document failure modes and mitigations. We test with real data before go-live.
Key controls to establish:
- Access: Role-based access with least privilege. No orphaned service accounts. All credentials rotated on a schedule.
- Data: Field-level data classification and masking. No export to unmanaged stores. Encryption in transit and at rest.
- Change: Version every prompt, model, and policy. Use pull requests and approvals. Maintain a change log.
- Audit: Immutable event logs for inputs, outputs, and writes to systems of record. Keep evidence for at least the retention period you use for the owning process.
- Risk: Document risk scenarios, for example incorrect price override, and a mitigation, for example human approval. Map each to a control owner.
Where do ERP and CRM integration fit?
They fit at the center, because value flows through those systems. We integrate at the API or message bus, not by scraping screens. We honor existing data contracts. We reuse your entitlement model. We write back with attribution, so audit shows what changed, when, by whom, and under what policy version.
Typical patterns:
- ERP integration: Use standard APIs or middleware connectors to read orders, invoices, and inventory. Use posted transactions or approved change APIs to write. Carry the acting identity. Add a reference to the AI service in free-text or custom fields where allowed for traceability.
- CRM integration: Expose the assistant as a widget or side panel in case and opportunity views. Read context from the current record. Write notes, tasks, and field updates through the CRM API with the agent’s identity. Log to CRM’s timeline.
- Eventing: Subscribe to business events from the integration layer. Emit decision events to the same bus so downstream systems can consume outcomes without polling.
Who owns what, and how do we staff it?
The business process owner owns the outcome and the SLA. The application owner owns the integration and data contracts. The platform or ML team owns the model stack, observability, and runbooks. Risk and security own the controls design and reviews. Support owns the pager. We do not ship until each owner signs their part.
A simple RACI:
- Process owner: Responsible for use case scope, metrics, thresholds, approval policies.
- App owner, ERP or CRM: Accountable for data access, API limits, change windows, and write-back safety.
- ML platform: Responsible for models, retrieval, prompts, versioning, deployment.
- Security and risk: Accountable for access, data handling, audit, and control tests.
- Support or SRE: Responsible for uptime, paging, runbooks, rollbacks, and incident response.
- Change advisory: Consulted on launch and major updates. Schedules and freeze periods.
- Legal and compliance: Consulted on data use and retention.
What telemetry do we need for observability?
We need to see inputs, decisions, outcomes, and costs in one place. That means logs, metrics, and traces that tie a business event to a model call and to a write in ERP or CRM.
At minimum, capture:
- Business key: Order ID, Case ID, Invoice ID.
- Identity: Acting user or service identity and entitlements applied.
- Model context: Prompt template version, model version, retrieval query, sources used.
- Decision: Recommendation type, confidence or rationale token count, policy flags tripped.
- Action: Fields changed, API calls made, success or failure, latency per step.
- Outcome: Human overrides, rework, error codes, customer impact if any.
- Cost: Token counts, API costs, compute minutes, per-decision totals.
Aggregate into dashboards:
- Accuracy proxy: Override rate by category and threshold band.
- Speed: End-to-end latency p50, p95, and time in queue.
- Stability: Error rates by dependency and time of day.
- Drift: Source coverage, retrieval hit rate, and change in top features over time.
- SLA: Percentage within policy thresholds, escalations triggered, backlog impact.
How do we write runbooks that survive 2 a.m.?
We write them as step-by-step playbooks with commands and screenshots, built from real incidents in testing. We separate triage from deep debug. We precompute safe fallbacks. We include exact paths to disable an integration or roll back a prompt version without waking five teams.
Runbook essentials:
- Triage checklist: What to check first, in order, with time limits. For example, verify ERP API status, then model endpoint health, then router queue depth.
- Known issues: Symptoms, root cause, and quick fix. For example, token limit exceeded on large case summaries, fix by switching to short template and requeue.
- Rollback: Commands or UI path to pin to last known good model or prompt. Document impact.
- Bypass: How to route traffic to manual processing. How to notify teams and customers if SLAs risk breach.
- Evidence: How to capture logs and screenshots for audit and post-incident review.
What changes between a lab pilot, a live-path pilot, and production?
The differences are ownership, risk posture, and blast radius. In the lab, we learn. In the live path, we earn trust. In production, we carry the SLA.
| Dimension | Lab Pilot | Live-Path Pilot | Production |
|---|---|---|---|
| Data access | Static export, anonymized where possible | Read from systems of record, scoped cohort | Read and write to systems of record under role-based access |
| Integration | None or mock | API read, controlled write via human-in-the-loop | Full API read-write with policy gates and backpressure |
| SLA | Best effort | SLO for pilot cohort, rollback defined | Contracted SLA with paging and on-call |
| Observability | Basic logs | Structured logs, dashboards | Full logs, metrics, traces, alerting, anomaly detection |
| Governance and controls | Ad hoc | Documented thresholds, approvals, audit events | Formal control library, evidence, and periodic tests |
| Runbooks | Not written | Drafted and tested in dry runs | Maintained, versioned, and exercised |
| Ownership | Project team | Named service owners | Operational owners with RACI and budgets |
| Change management | None | CAB notified, feature flags | CAB approved, change windows, release notes |
| Rollback | Revert code | Router switch, version pin | Instant rollback with tested playbook |
Can we see a worked example in the live path?
Yes, here is a narrow, measurable case. We pick invoice dispute triage in order-to-cash. Goal: cut resolution time without raising credit risk. Systems: ERP for invoices and credits, CRM for cases and customer communication.
Day 0 to 14, design and controls:
- Define scope: Consumer disputes under a fixed dollar threshold, no legal holds, three regions, business hours only.
- Ownership: Finance ops owns outcome. ERP owner approves write-back. CRM owner approves case updates. Platform owns model and observability. Risk approves controls.
- Access: Service identity with read to invoices and disputes, write to CRM comments, draft-only credit memos in ERP. No write to GL.
- Policy: Under threshold, model drafts credit memo with reason code and supporting references. Human approves and submits. Over threshold, model drafts response only.
- Observability: Instrument business key, prompt version, retrieval sources, draft fields, approval outcome, and latency per step.
- Runbooks: Draft rollback, bypass to manual queue, and frequent failure fixes.
Day 15 to 30, live-path pilot with guardrails:
- Volume: 10 percent of eligible disputes, random assignment. 90 percent remain baseline.
- Dual run: Model drafts in ERP, human approves in ERP. Model updates CRM case with rationale and links to policy. All under the agent’s identity.
- Thresholds: Auto-approval disabled. Any change requires human submit. Flags for certain SKUs and accounts block drafts.
- Metrics: Track average handle time, first-contact resolution, override rate, and credit leakage. Latency target under five seconds p95 for draft creation.
- Review: Daily huddle on overrides. Weekly risk review on leakage range, for example expected increase of 0 to 0.2 percent of volume, tied to mix.
Day 31 to 60, expand safely:
- Threshold change: Allow auto-submit under a lower dollar threshold where override rate sits under a set percentage for two weeks.
- Coverage: Add a second region. Keep volume cap at 20 percent total.
- Controls: Add model version pinning and prompt A, B test with approval gates.
- Training: Short sessions for agents. Show exactly what changes and how to override.
Day 61 to 90, decide to scale or stop:
- Gate criteria: SLA met for pilot cohort for four consecutive weeks. Override rate below the target. Credit leakage within risk tolerance. No control failures.
- If green: Increase volume, keep thresholds, and start change process for full production.
- If red: Roll back to draft-only or pause and remediate. Use evidence from observability to target fixes.
Outcome pattern we see: The model saves minutes per case by assembling context and drafting the memo. Humans still make the call on edge cases. The ERP and CRM remain the single source of truth. Audit holds. The pilot touches real work from day one without raising unacceptable risk.
How do we measure value without inflating claims?
We measure the parts we can control and show the mechanism. We do not guess at attribution across a whole quarter. We use cohort comparisons and operational metrics.
Useful measures:
- Time saved per case: Before and after for the pilot cohort, measured in the system, not by survey. Range depends on case mix and system latency.
- Rework rate: Percentage of cases reopened post-credit. Should stay flat or drop.
- Override rate: Percentage of model drafts changed by humans. Drives where to tighten prompts or rules.
- Latency: p95 end-to-end time for draft creation. Directly impacts throughput.
- Adoption: Percentage of eligible cases processed through the service without bypass.
Tie each measure to a decision. If override rate is too high on a code, update the retrieval or tighten a rule. If latency spikes at 9 a.m., add capacity or precompute summaries.
What are the non-negotiables before we start?
- A named workflow in ERP or CRM where the service will sit.
- Read and write paths agreed by the application owners.
- Role-based access and a service identity provisioned.
- Observability plan with logs, metrics, and traces.
- Runbooks drafted and dry-run tested.
- A rollback switch in the router.
- RACI agreed and owners scheduled for on-call.
- Change window booked and CAB informed.
If any of these are missing, we are not piloting, we are demoing.
What changes when we go multi-use-case?
Coordination and standards matter more. We stop treating each use case as a bespoke build. We build common services for retrieval, identity, prompt templates, policy enforcement, and logging. We enforce a shared release cadence and a shared feature flag system. We version everything across use cases. We centralize model registry and prompt library. We keep the domain logic close to the owning team.
At this stage, we also rationalize run costs. We set per-request budgets and alerts. We move heavy work to batch where latency does not buy value. We cache summaries. We right-size models per task rather than defaulting to the largest one.
What if our data is not perfect?
We ship anyway, but we constrain scope. We start with use cases that tolerate uncertainty and where humans already do reconciliation. We use retrieval from systems of record instead of building a separate corpus. We log sources and highlight gaps so the data team has a punch list. We improve data as a byproduct of the pilot, not a blocker to it.
How does change management actually look here?
We run change like any other operational change. Early stakeholder mapping. Clear “what changes for me” for agents. Training in the workflow, not in slides. Office hours the first two weeks. Fast feedback loops. We document wins and misses with screenshots of the real system. We keep the scope honest. We avoid adding new process steps unless we delete at least one old one.
Bottom line: what clears the production bar?
Clear ownership, ERP and CRM integration, governance and controls, observability, runbooks, and a live-path pilot with guardrails clear the bar. Treat AI as an operational change, not a tooling experiment. If we cannot name the system of record we will write to and the runbook we will follow at 2 a.m., we are not ready. If we can, the model will have a shot, and the pilot will not die in the lab.
