The Real Cost of an AI Workflow

The central idea

The economically useful unit is a completed, acceptable task, with review and failures included.

In this essay +

1. Define what counts as finished
2. Build a cost model around that definition
3. A numerical example with visible assumptions
4. Compare with the simplest credible alternative
5. Include the costs that do not arrive on an invoice
6. Measure a pilot rather than extrapolating a demo

A model call costs a few cents. A research team completes thousands of document tasks. Multiplying the two can make the business case look obvious, but it leaves out the part of the workflow where most of the decision may live: reviewing, correcting, and trusting the result.

The right comparison starts with a clearly defined task and an existing baseline. Without those, “cheaper AI” is a statement about an input price rather than an economic outcome.

Define what counts as finished

Consider a narrow fictional task: extracting a small set of reported financial metrics from an annual report, with source locations and no missing required fields. A successful task is a record that a reviewer accepts under that specification.

The same output format is not automatically acceptable for every use. A draft for internal research and a figure sent to a client may require different review. If the required standard changes halfway through the comparison, the cost figures stop describing the same product.

Write the acceptance criteria before measuring model performance. Include accuracy, evidence, latency, and the handling of missing or ambiguous data. “Looks plausible” is not an operational definition.

Build a cost model around that definition

A basic accounting identity for a batch is:

cost per accepted task =
  (inference + retrieval + infrastructure + review + rework + overhead)
  / number of accepted tasks

Expected downstream error cost can be modeled separately when there is a defensible basis for estimating it. Adding an invented precision to a rare, severe loss is worse than showing a range and acknowledging what is unknown.

Retries belong in the numerator. So do tasks that are reviewed and then rejected. Counting only successful calls understates the cost of the actual process.

A numerical example with visible assumptions

Suppose a batch contains 1,000 tasks. These are deliberately invented inputs, not vendor prices or measured performance:

Cost component	Assumption	Batch cost
Model and retrieval	$0.04 per attempted task	$40
Review	2 minutes per task, $30 per hour	$1,000
Extra rework	100 tasks, 5 extra minutes each, $30 per hour	$250
Allocated infrastructure and overhead	Assumed for this batch	$100
Total	Before downstream error costs	$1,390

If 950 tasks are ultimately accepted, the measured cost is about $1.46 per accepted task. The inexpensive call was a small part of the total.

Now imagine a model that adds $0.06 per attempted task but reduces review by 30 seconds on every task, with every other assumption unchanged. It adds $60 in inference and saves $250 in review, producing a $190 batch saving. Whether it really reduces review, preserves quality, or changes the acceptance count requires an experiment.

The calculation illustrates sensitivity. It is not evidence that a more expensive model will pay for itself.

Compare with the simplest credible alternative

The baseline might be a manual process, a spreadsheet import, a rules-based extractor, or a document template. A general-purpose model should earn its complexity against the alternative that actually fits the task.

For repeated work on a fixed table format, a deterministic parser may be easier to inspect. For messy narrative disclosures, language understanding may contribute more. A hybrid workflow can route simple cases through rules and ambiguous cases to a model or a person.

Routing changes the evaluation population. If the model receives only the difficult cases, comparing its raw success rate with an earlier all-task average is misleading. Keep the case mix visible.

Include the costs that do not arrive on an invoice

Operational work includes maintaining evaluation sets, checking new model versions, investigating failures, and updating source integrations. Changes in a model or prompt can alter the system even when the interface looks unchanged.

NIST’s Generative AI Profile provides a broader voluntary risk-management reference. The cost worksheet here is an original decision tool, not a claim that NIST prescribes this accounting method.

For a small deployment, record the minimum work needed to detect a regression and recover from a failed run. If the business case requires eliminating the review needed to establish quality, the proposed savings may be internally inconsistent.

Measure a pilot rather than extrapolating a demo

Sample ordinary cases, difficult cases, and cases where the correct result is to abstain. Measure accepted outputs, review minutes, rework, and failure types using the same standard as the baseline. Run a sensitivity analysis for lower volume, higher review time, and a less favorable acceptance rate.

A demo asks whether the system can produce a good answer. An economic evaluation asks how often it does so, what it takes to recognize that answer, and what the entire process costs.

That is the number worth optimizing. A cheap model call is useful only when it is part of an acceptable completed task.