Internal Coherence Maximization: The Research Behind AI That Doesn't Drift
By Lexi Banks · · AI Strategy
Enterprise AI rarely breaks — it drifts. A look at Internal Coherence Maximization (ICM), the unsupervised method that beat human supervision, and how we apply the coherence principle as automated quality gates.
Key takeaways
- AI systems in production usually degrade through drift, not failure — and one-at-a-time review structurally cannot detect it.
- ICM optimizes for mutual predictability and logical consistency, matching golden-label training and beating crowdsourced human supervision.
- Coherence is measurable without ground truth, which makes it usable where production systems have no answer key.
- Consistency has to be an automated gate in the publish path, not a review step someone remembers to run.
- Optimizing for agreement alone converges on saying nothing consistently — coherence still needs a human-set standard for quality.
Most enterprise AI doesn't fail loudly. It drifts.
The pilot works. The first fifty outputs are good — you read every one of them. Then volume goes up, nobody is reading every output anymore, and six weeks later a customer forwards you something that doesn't sound like your company at all. Nothing broke. No error fired. The system just slowly stopped agreeing with itself.
This is the scaling problem nobody puts on a slide: quality control in AI systems is usually a human, and humans do not scale. The reviewer becomes the ceiling. Worse, the reviewer becomes the standard — and if they're inconsistent, tired, or new, the system inherits that.
A 2025 paper co-authored by researchers at Anthropic, NYU, George Washington University, and Schmidt Sciences takes a genuinely different swing at this problem, and the idea underneath it changed how we think about building content and automation systems that run without a person in the loop.
What Internal Coherence Maximization actually is
The paper is Unsupervised Elicitation of Language Models. The method is called Internal Coherence Maximization, or ICM.
The setup: you want to train a model on a task, which normally means you need labeled data. Labels come from humans. Humans are slow, expensive, inconsistent, and — on hard enough tasks — worse than the model you're trying to train.
ICM's move is to skip the humans. Instead of asking "what does a labeler say the answer is?", it asks the model to find a set of labels that hangs together. Specifically, it optimizes for two things at once:
Mutual predictability. Given every other labeled example, how well can the model predict this one? If a set of labels all reflect a single coherent concept, each one should be inferable from the rest. If a label is arbitrary or wrong, it will sit badly against its neighbors. The paper identifies this as the primary driver of the method's performance.
Logical consistency. The labels must not contradict each other. On its own, this is not enough — you can satisfy consistency by labeling everything the same way — so it works as a regularizer that stops the search from collapsing into a trivially "consistent" but useless answer.
Finding the genuinely optimal label set is an integer programming problem, which is computationally hopeless at real dataset sizes. So ICM uses an approximate search inspired by simulated annealing: start from a handful of randomly labeled examples, add one at a time, fix inconsistencies as they appear, and accept or reject each move based on whether it improves the score.
The results are the part that should get your attention. Across GSM8K-verification, TruthfulQA, and Alpaca reward modeling, ICM matched training on golden labels and beat training on crowdsourced human supervision. On a superhuman capability elicitation task — where the model plausibly knows more than the annotator — it hit roughly 80% accuracy against an estimated 60% for humans.
Read that again. Removing the human labels made it better.
The insight that transfers to production systems
You are probably not fine-tuning a frontier model this quarter. That's fine — the transferable idea isn't the algorithm, it's the premise underneath it:
Coherence is a measurable signal, and you can measure it without ground truth.
That is a big deal for anyone running AI in production, because ground truth is exactly what you don't have. When your system generates a blog post, drafts a reply, scores a lead, or routes a ticket, there is no answer key. You can't diff it against the correct output, because nobody knows what the correct output was.
But you can ask whether it agrees with everything else the system has produced. And that question is answerable, automatically, at any volume.
This reframes quality from a review problem into a systems problem. Instead of "did a human approve this?", the question becomes "is this consistent with the set?" — and the second question is one a machine can ask ten thousand times a day.
How we apply the principle
Straight up: we are not running the ICM algorithm. ICM is a training-time method for eliciting labels from pretrained models, and that is not what a content or automation pipeline does. Anyone telling you they've "implemented ICM" in a marketing product is selling you a paper title.
What we did take is the discipline: treat self-consistency as a first-class, automated quality gate rather than a thing a person eyeballs at the end. In our own content engine, that shows up as a stack of checks that run before anything reaches a client site:
Judged against the set, not alone. Every generated post is scored for semantic diversity against everything already published. A piece that substantially restates existing work fails the gate, even if it's well written on its own terms. This is the check that's closest in spirit to ICM — the unit of evaluation is the corpus, not the document.
Brand coherence as a hard gate. A separate review pass asks whether the output is actually on-topic and on-brand for the client it's being written for. Not "is this good writing" — "does this cohere with what this company is." Off-brand output gets flagged for a human instead of shipped.
Originality and structural checks. Length, excerpt quality, and originality thresholds are enforced in code. Work that fails is saved as a draft and never published. We had a real example of this recently: a malformed topic produced a post with zero body content, and the self-check caught it and refused to publish. That is the system working.
Client rules as code, not prompt suggestions. Banned phrases, claims that must never be made, required disclaimers, compliance profiles for regulated industries — these live as enforced constraints in the pipeline, not as polite requests buried in a system prompt. A prompt is a suggestion. A gate is a guarantee.
None of this is exotic. That's the point. The leverage isn't in a clever algorithm — it's in deciding that consistency gets checked every time, by the system, instead of sometimes, by whoever is around.
What to do if you're running AI in production
Four things, in rough order of payoff:
Evaluate outputs as a set. Most teams review outputs one at a time, which structurally cannot detect drift. Drift is only visible in the relationship between outputs. Sample across time and compare.
Make consistency a gate, not a review step. If a check only runs when someone remembers to run it, it is not a control. Put it in the path where failure blocks the publish.
Instrument drift as a metric. You want to find out from a dashboard that outputs are diverging, not from a client. Contradiction rate and similarity-to-corpus are both cheap to compute.
Encode your rules where they're enforced. Anything you'd be embarrassed to see in front of a customer belongs in code, not in a prompt.
The honest limits
ICM is not a quality-assurance product, and it isn't magic. The paper is careful about this and so should we be:
- It works by eliciting knowledge the pretrained model already has. It cannot conjure a concept the model doesn't represent — if the model has no internal notion of your brand voice, coherence maximization has nothing to maximize.
- The optimization is approximate. The exact objective is computationally infeasible, so results depend on the search behaving well.
- Mutual predictability does the heavy lifting; logical consistency alone is easy to game. A "consistency check" that just confirms outputs don't contradict each other can be satisfied by outputs that are uniformly bland — which is its own failure mode.
That last point is worth sitting with, because it's the trap in applying this naively. A system optimized only for internal agreement will happily converge on saying nothing, consistently. Coherence has to be paired with a standard for what good actually looks like — which, for us, is where the human still belongs: setting the bar, not inspecting each unit of output against it.
The takeaway
The reason this research matters outside the lab isn't the benchmark numbers. It's the demonstration that a system's own consistency carries real signal about its quality — enough signal, in the paper's setting, to beat human supervision outright.
If you're deploying AI at any volume, that should change where you spend your effort. The question isn't how to review more output. It's how to build a system that notices, on its own, when it has stopped making sense.