Build AI Into the Operations You Already Run

By Kalyxi · 2026-10-05

Skip bolt-ons. Build AI into existing systems, queues, and approvals. Start with exception-heavy work, control risk, and ship production value fast.

Where should we use AI first in operations?

We should start with exception-heavy workflows where the outcome is measurable and the work already routes through a queue. Examples: AP invoice coding with missing PO lines, order holds from credit checks, carrier claim adjudication, warranty approvals, and IT ticket triage. These queues have clear states, known SLAs, and labeled outcomes like approved, denied, routed to vendor, or needs buyer review. That gives us a ground truth for evaluation. It also anchors the project in the operational calendar: month close, ship date, claim aging.

A simple litmus test: if a supervisor can train a new hire to resolve 70 percent of items in the queue using a checklist and past examples, we can capture that knowledge with retrieval and structured prompts. If they need three months of side-by-side ramp to read context hidden across six systems, we can still do it, but we start by aggregating context and standardizing decisions before we add a model.

What is the mechanism, not the demo?

The mechanism is event-level enrichment inside the system of record. When an item enters a queue, we attach the right context, call a model, and write back a proposed resolution with confidence and rationale. Humans approve, edit, or reject. The system logs the decision, updates metrics, and learns what to automate next. We do not forward emails to a side address. We do not paste screenshots into a chat. We instrument the actual work object.

Here is what that looks like:

• Trigger: a new item enters a queue in SAP, ServiceNow, Salesforce, or your custom system.
• Context: we fetch related records from the system of record and approved sources, for example the vendor master, PO, prior invoices, credit limits, contract terms, knowledge articles, and recent communications.
• Decision: we form a structured task, call a fit-for-purpose model, and get back a proposed classification, draft, or action.
• Control: we apply policy rules, role checks, and thresholds. If the action moves money or changes customer status, we require an approval. If not, we can auto-apply.
• Writeback: we record the result, rationale, inputs, and model version on the work item. We update metrics and dashboards.

This mechanism aligns with how auditors think. Identity sits in your IdP. Permissions remain in your systems. Approvals follow your existing flows. The model is a function inside the process, not a shadow system.

Which models should we use, and how do we choose?

We should pick models by task, latency, and cost, not by leaderboard. For classification and extraction on structured documents, small instruction-tuned models often win on latency and price. For long context and nuanced drafting, larger context windows or specialized document models help. For tasks with privacy needs, a self-hosted or private endpoint is sensible.

A workable routing plan:

• Default to a fast 7B to 13B model for routing, tagging, and short drafts.
• Use a stronger general model for long-form drafting and edge cases.
• Use a document model for OCR-heavy or table-heavy PDFs.
• Cache frequent prompts and recurrent instructions.

We keep a simple policy table: task type, input length, risk level, max latency, max cost, allowed models. That controls drift from ad hoc choices and gives procurement and security a handle to review.

Should we start with retrieval or fine-tuning?

Start with retrieval. Retrieval cuts hallucination by grounding the model in your contracts, policies, and past decisions. We build a thin index over the sources of truth, chunk in a way that respects document structure, and embed with a stable model. We then pass only what the task needs. Fine-tune only when variance stays high after retrieval and prompt work.

Fine-tuning makes sense when:

• The label space is stable and unambiguous, for example five to ten categories with examples in the thousands.
• The process language is unique and your prompts remain long to teach that language each time.
• You want consistent style in outbound drafts where brand voice is strict.

Retrieval is better when:

• Policies change monthly and you cannot afford to re-train that fast.
• Decisions must cite the exact clause or ticket.
• You need cross-system joins at inference time.

How do we integrate with SAP, ServiceNow, Salesforce, and others without breaking them?

We integrate at the API and event layer. We subscribe to queue events, we read and write through supported APIs, and we respect rate limits. We avoid brittle UI automation. If a vendor exposes a document classifier or a knowledge search, we can call it, but we still keep our own retrieval for cross-system context.

A practical pattern:

• Use webhooks, message queues, or a change data capture stream to catch new items.
• Cache lightweight entity profiles, for example a vendor profile with top fields and recent risk flags, to avoid repeated wide reads.
• Write back to a designated field set or related object for the proposed decision and rationale. Keep the human in the native screen.
• Log every request and response with model ID, prompt hash, and latency, then link the log to the work item.

We keep the system of record as the source of truth. We do not build a parallel task tracker. We build thin, durable glue.

What do we measure, and how do we prove value?

We measure what operations leaders already track. That keeps finance aligned and politics quiet. The core set:

• Cycle time per item and per stage.
• First-pass yield and rework rate.
• Human minutes per item and per exception type.
• SLA attainment and backlog age distribution.
• Error rate on outcomes that matter to money or customer status.

Quality measurement needs a rubric. For classification, accuracy and confusion across classes. For drafts, a scaled rubric with three to five criteria, for example completeness, tone, compliance, and actionability. We score with a rotated human panel on a random sample. We do not grade our own homework with the same model unless we also verify a human subset.

Expect ranges, not certainties. On exception-heavy queues, we see 20 to 60 percent cycle-time cuts when the model pre-classifies or drafts and a human approves. Cost savings fall between 15 and 40 percent when volume is high and approval rates rise over time. The range depends on starting data quality, integration depth, and how strict your approvals are.

Where do we put humans, and when do we remove them?

We put humans at decision points where the impact is high or the model’s confidence is unstable. We remove them where the action is reversible, the cost is small, and the model has shown consistent performance.

Common thresholds:

• Money movement: always require approval above a set amount.
• Customer status changes: approval until three months of stable performance and a post-change survey metric holds.
• Internal routing or tagging: auto-apply if confidence is above a set threshold and auto-rollback exists.

We design the approval UI to add value. Show the rationale, the exact cites from contracts or tickets, and the diffs against policy. Make approve the default for high-confidence items, but make override cheap. Every override trains the next decision.

What about data privacy, security, and audit?

We route identity through your IdP. We fetch only fields we need. We encrypt at rest and in transit. We keep PII minimization rules in the policy layer and mask before prompts when the model does not need the PII. We store prompts, responses, and retrieval snippets for audit with retention policies that match your system of record.

For vendor risk, we keep a catalog of allowed models with their data handling statements and regions. We choose private endpoints or self-hosted options when data residency demands it. We log model names and versions on each decision. That makes audits repeatable.

Build or buy: what is worth owning?

We should own the process knowledge, policy engine, and integration glue. We should not build foundation models. We should choose hosted or self-hosted models based on data and latency constraints.

Here is a simple comparison:

Ownership here means we control configuration and can switch vendors without redoing policy. Buying hosted means we take a service with SLAs and data controls. Using vendor features means we enable embedded functions in SAP, Salesforce, or ServiceNow for simple cases.

What does a worked example look like in accounts payable?

Operational reality: invoices arrive by email and EDI. Some match cleanly. Many do not. Reasons: quantity variances, stale vendor master, freight lines off contract, tax handling. A team triages exceptions, reads POs and contracts, emails buyers, and codes lines.

Mechanism in six steps:

1. Trigger: an invoice enters the AP exception queue in SAP.
2. Context: we pull the PO, receipts, vendor profile, last three invoices, contract clauses on freight and tax, and any open disputes.
3. Decision: a model classifies the exception type and drafts a proposed resolution: accept variance within tolerance, short pay freight, route to buyer, or request updated invoice.
4. Control: if short pay exceeds a set amount, we require buyer approval. If within tolerance, we auto-post.
5. Writeback: we log the classification, rationale with contract cites, and action taken on the invoice object.
6. Measure: we track cycle time, first-pass yield, and number of emails sent.

Expected results after stabilization:

• 30 to 50 percent reduction in time to post exception invoices because classification is instant and drafts remove back-and-forth.
• 20 to 40 percent fewer buyer touches because policy routing handles routine variances.
• Error rate held within existing thresholds because high-impact actions still require approval.

Costs: integration work usually runs two to six weeks depending on SAP module customization. Model costs stay small relative to labor. A typical invoice flow might spend fractions of a cent to a few cents per item on model calls, depending on document length and retries. The main cost sits in integration and change management.

What about customer email triage in Salesforce or Zendesk?

Operational reality: support inboxes mix order status, returns, bugs, and billing. Many responses repeat policy language. Agents search knowledge articles and copy templates.

Mechanism:

• Trigger on case creation.
• Retrieve order data, entitlement, recent interactions, and relevant knowledge articles.
• Classify intent and sentiment. Draft a reply with specific order details and action steps. Suggest next internal task, for example create RMA or escalate to tier two.
• Respect entitlements, do not promise actions outside SLA. Always show the draft to the agent. Auto send only for low-risk intents.

Expected results:

• 20 to 40 percent faster first response for common intents.
• Higher consistency on policy language.
• Agent minutes per case down by 15 to 35 percent on repeatable topics.

How do we handle latency, retries, and failure modes?

We design for production, not demos. That means timeouts, retries with backoff, fallbacks, and clear user feedback. We set a strict per-task latency budget. If a model cannot return in time, we fall back to a cached template or skip the assist. We batch work where possible but keep human-facing steps snappy.

Failure modes to plan for:

• Model timeout: show a simple assist or skip with a log.
• Bad retrieval: detect poor coverage and widen the query or ask for a human note.
• Policy conflict: stop and route to an approver with the conflict explained.
• Vendor outage: route to an alternate model or queue for later.

We monitor token usage, latency, and error rate. We alert when metrics breach thresholds. We keep a playbook for failover.

How do we prevent prompt sprawl and brittle setups?

We keep prompts versioned and modular. We write task prompts with clear input and output contracts, for example JSON with a schema. We keep system instructions short and process specific. We lint prompts and test them with fixtures. When prompts change, we run a regression set of past cases to compare outcomes.

We also remove secrets from prompts. We pull dynamic policy from a single source, not from text baked into prompts. We avoid clever tricks that do not survive scale.

Do we need an LLMOps stack, and if so, what parts matter?

We need the parts that keep models in control. The core set:

• Prompt and policy versioning with approvals.
• Offline evaluation with test sets and spot checks.
• Online monitoring for drift and bias.
• Model catalog with allowed uses and PII rules.
• Experiment tracking tied to business metrics, not just loss curves.

We do not need twelve dashboards. We need the ability to answer three questions: what decided this, who approved it, and how often does it work.

How do we select vendors without getting stuck?

We pick vendors we can swap. That means we separate policy and routing from the model provider. We use open formats for embeddings and stores where practical. We prefer SDKs that do not embed opinionated state into our process definitions. We ask for clear data handling terms, residency options, and rate limit transparency.

We also lean on existing platforms where it helps. If your team lives in ServiceNow, using its virtual agent for simple intents can make sense. For complex cross-system context and policy-heavy actions, we keep control in our own orchestration and call back into ServiceNow for the writeback. The same logic applies to SAP and Salesforce.

What is the recommended starting plan for a mid-market or enterprise team?

We recommend a 90-day plan with one workflow to production.

Weeks 1 to 3:

• Pick a queue with 5,000 to 50,000 items per month and a clear outcome label.
• Map states, approvals, and policy. Pull a sample of 200 items with outcomes and notes.
• Build retrieval over the documents and records that matter, no more.
• Define metrics and a quality rubric.

Weeks 4 to 6:

• Integrate triggers and writeback with the system of record.
• Ship classification and draft-in-UI for human approval.
• Start logging and dashboards.

Weeks 7 to 9:

• Raise automation on low-risk actions with confidence thresholds.
• Tune prompts and retrieval based on error analysis.
• Add a stronger model for the 10 percent hardest cases if needed.

Weeks 10 to 12:

• Formalize approvals and policy in code.
• Run side-by-side on a full cohort.
• Train supervisors to review logs and adjust thresholds.

Week 13:

• Move to production on a defined slice.
• Set a monthly change window for prompts and policy.

What pitfalls should we avoid?

• Starting with a greenfield bot outside systems of record. It will not survive audit or scale.
• Chasing a model upgrade without fixing retrieval or policy gaps.
• Hiding model outputs in a separate UI. Put drafts and decisions where work already happens.
• Skipping evaluation. A demo that looks right can fail repeatably on edge cases.
• Forgetting latency. Agents will abandon assists that add two seconds per action.

When should we fine-tune, and what does it cost?

Fine-tuning is a phase two move. Use it when retrieval and prompt changes stabilize but variance remains. Costs vary by provider and data size. Budget from low five figures to mid five figures for an initial run including data prep and evaluation, plus ongoing runs when policy or product shifts. The return shows up if you send high volume through the tuned task, for example millions of classifications per month or thousands of long drafts where tone must be exact.

What about documents, OCR, and forms?

Treat OCR and form parsing as utilities. Use specialized services that handle skew, stamps, and tables. Post-process with a model for classification and reconciliation against known fields. Keep a golden schema for each document type and validate. Do not ask a general model to read a poor scan without help.

How does Kalyxi fit?

We build AI into the operations you already run. We attach to your queues, pull only the context each task needs, and route decisions by policy. We write back into the systems your teams use every day, with approvals where your auditors expect them. We start with retrieval, add models that meet your latency and risk profile, and measure value with the same metrics your COO tracks.

Our recommendation is simple: do not add another tool on top. Put models inside the flow of work you already operate. Start with one exception-heavy workflow, ground it in your data, keep humans where risk is real, and move volume only when the numbers hold. That is how AI holds up in production.