---
title: "AI-Driven Dispatch Learns From What You Did, Not From What You Should Have Done"
id: "26507"
type: "post"
slug: "ai-driven-dispatch-counterfactual-training-data-2026"
published_at: "2026-09-10T14:30:00+00:00"
modified_at: "2026-09-10T20:23:09+00:00"
url: "https://locus.sh/blogs/ai-driven-dispatch-counterfactual-training-data-2026/"
markdown_url: "https://locus.sh/blogs/ai-driven-dispatch-counterfactual-training-data-2026.md"
excerpt: "Dispatch history records one executed plan per cycle and zero alternatives. What learns from execution, what cannot, and how to instrument the difference."
taxonomy_category:
  - "General"
---

#### [General](https://locus.sh/blogs/category/general/)

# AI-Driven Dispatch Learns From What You Did, Not From What You Should Have Done

[Aseem Sinha](/author/aseem_locus/)

Sep 10, 2026

17 mins read

AI-driven dispatch assigns orders to vehicles against live constraints and improves as operating data accumulates. That improvement is real, but it is not uniform, and the distinction matters more than the claim. Two very different things live inside the phrase “learns from history.” Travel time on a road you drove, service duration at a stop you visited, geocode error at an address you delivered to: these are properties of the world, observed on every execution regardless of what you decided, and they accumulate genuine labels. Whether a different assignment would have been better, whether the load you deferred would have made it, whether another driver would have been faster: these are properties of decisions you did not take, and they are never observed at all.

Each dispatch cycle produces exactly one executed plan and zero measured alternatives. Five years of history at 200 plans a day is 365,000 executed plans and still zero observations of any alternative to any of them. Volume does not fix that, because the problem is not sample size but that the label you would need was never generated.

## Key Takeaways

- Dispatch data is censored by construction. Every cycle yields one outcome for the chosen action and nothing for the alternatives, no matter how long you run.
- World parameters are learnable from execution. Policy quality is not, unless you deliberately instrument for it.
- Dispatcher overrides break the measurement. At a 15% override rate concentrated on hard cases, an engine failing on 40% of hard plans and 5% of easy ones measures at 5.0% against a true 10.25%, understating itself 2.05x.
- The perverse consequence: the better your dispatchers are at catching bad plans, the better the engine looks and the less it can learn.
- Travel-time learning is biased toward routes your current policy already likes. Arcs you never traverse never get measured, so the blind spot reinforces itself.
- Running 2% of plans on a second-best option yields 1,460 labeled alternatives a year against zero today.

## Why dispatch history is the wrong training set for allocation policy

The claim that an allocation engine improves with data is well founded and easy to demonstrate. Year-two plan quality genuinely beats year one in most deployments. What is rarely examined is which part of the model improved.

Consider what execution records. A vehicle ran 40 stops, and you know the departure time, the arrival and dwell at each stop, the distance covered and whether each window was met. Every one of those is a measurement of the physical world under conditions you happened to create, and feeding enough of them back makes your travel-time matrix, service-time distributions and address database materially better. This is why [route optimization engines improve with deployment data](https://locus.sh/blogs/how-the-worlds-best-route-optimization-engine-works/)
 and why the second year of a rollout outperforms the first.

Now consider what execution does not record. The solver evaluated a large number of candidate assignments and returned one. The other candidates were discarded without being run, so their costs remain estimates produced by the same model you are trying to evaluate. When you ask whether the engine allocated well, you are asking a question about the alternatives, and the alternatives have no observations attached. In statistical terms the outcome data is censored: you see the result of the action taken and nothing else. Standard supervised learning cannot resolve this, because there is no label to fit.

The stake is not abstract. ATRI’s 2026 cost report puts the industry-average cost of operating a truck at [$2.336 per mile in 2025](https://truckingresearch.org/2026/07/new-atri-report-details-accelerating-costs-and-low-profitability-despite-cuts/)
, a record for the series and 3.4% above the prior year, with $1.854 excluding fuel and the sharpest rises in tolls at 13.2% and repair and maintenance at 8.6%. Against that cost base an allocation policy that is quietly 5% worse than an available alternative is expensive, and the reason it stays quiet is that the alternative was never run.

This is a solved problem in outline, if not in ease. The canonical treatment comes from ad placement, where [Bottou and colleagues showed that counterfactual estimates can be recovered from a live learning system](https://arxiv.org/abs/1209.2355)
 by logging the randomization already present in serving decisions and reweighting observed outcomes accordingly. The approach transfers to dispatch. The prerequisite does not come for free, because it requires the system to record what it considered, and with what probability it chose, at the moment of choosing.

| Also Read: Agentic TMS Capabilities for Logistics Automation |
| --- |

This is not an argument against AI-driven dispatch. It is an argument for knowing which claim you are buying. “The system will predict travel times more accurately over time” is supportable and measurable. “The system will discover better allocation policies from your history” requires instrumentation most deployments lack, and asking which a vendor means is a five-minute question.

| What execution observes | What it never observes |
| --- | --- |
| Travel time on arcs actually traversed | Travel time on arcs the policy avoids |
| Service duration at stops actually visited | Service duration at stops assigned elsewhere |
| Geocode accuracy at delivered addresses | Accuracy at addresses never attempted |
| Whether the executed plan met its windows | Whether a different plan would have met more |
| The cost of the route that ran | The realized cost of the route that did not |
| That a deferred load was deferred | Whether the deferred load would have made it |

The right-hand column is where allocation policy lives. That is the whole problem in one table.

## How to instrument dispatch so the learning loop can close

None of this is unfixable, but it requires generating the missing labels rather than hoping they emerge from volume.

### 1. Log the alternatives, not only the choice

When the solver returns a plan, it has already evaluated others and priced them. Persist the top candidates alongside the selected one, with their predicted cost and the constraint that separated them. This costs storage and nothing else, and it turns every cycle from a single data point into a ranked set. Without it there is no record that a decision was even a decision.

### 2. Store the prediction next to the outcome

The one comparison always available is predicted versus actual on the plan you ran: predicted arrival against actual, predicted service time against actual, predicted route cost against settled cost. This measures the model rather than the policy, a narrower question but an answerable one, and prediction error is where most allocation error originates anyway. An engine confidently wrong about travel time allocates badly for reasons unrelated to its allocation logic.

### 3. Treat every dispatcher override as a labeled example

An override is the cheapest counterfactual an operation will ever get. Someone with local knowledge looked at a machine-generated plan, judged it wrong, and produced an alternative that then ran. That is a labeled pair: rejected plan, executed plan, observed outcome. Most operations discard it, overwriting the original with the final plan. Keep both plus a reason code and the override log becomes the highest-value training data in the building.

| Also Read: Top 8 AI-Driven Dispatch Companies for Enterprise Logistics |
| --- |

### 4. Run deliberate exploration on a small share of plans

To learn whether the second-best option was actually worse, some of them have to run. Reserving a small fraction of low-risk plans for a deliberate alternative generates labels the rest of the system cannot. At 200 plans a day, 2% is four plans daily and 1,460 labeled alternatives a year, against zero under a pure exploitation policy. The reservation has to be chosen for low consequence rather than at random, and it has to be logged as exploration so nobody reads it as an error.

### 5. Use natural experiments across sites

Networks with multiple similar depots already run a quasi-experiment whenever policy differs between them. Two depots with comparable order density, geography and fleet mix running different allocation rules produce a comparison no single-site history can. This is the cheapest counterfactual available at enterprise scale, and it requires only that the difference in policy be documented at the time rather than reconstructed afterwards. The discipline is established outside logistics: the authors of [Trustworthy Online Controlled Experiments](https://experimentguide.com/)
 describe Google, LinkedIn and Microsoft running more than 20,000 controlled experiments a year, and the recurring lesson is that the platform matters more than the statistics. Once a comparison costs almost nothing, teams run them. A 40-depot network has the units; what it lacks is the habit of recording which policy was live where.

### 6. Watch arc coverage, not just data volume

Travel-time learning is only as broad as the routes your policy chooses. If 30% of network arcs are traversed in a month, 70% carry no fresh timing data, and the engine is optimizing against a map it has measured in part. Where the untraversed arcs are avoided because of a stale estimate, the avoidance keeps the estimate stale. Reporting coverage alongside accuracy is what surfaces that loop. Google’s engineering account of production machine learning names the pattern directly, listing [hidden feedback loops](https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems)
 among the risk factors that make deployed models expensive to maintain. They are hard to spot precisely because the system keeps reporting healthy metrics while its inputs quietly narrow.

## What the override rate does to your performance measurement

The interaction between human oversight and machine measurement is the part that surprises operations teams, because both are virtues.

Assume dispatchers override the engine on 15% of plans, and that overrides concentrate on the hard cases, which is what a competent dispatcher does. Assume the engine would fail on 40% of those hard plans and 5% of the easy ones. If it ran everything, its true failure rate would be 0.15 times 0.40 plus 0.85 times 0.05, or 10.25%. But it does not run everything. The hard 15% was removed before execution, so the engine is measured only on the easy remainder, where it fails at 5.0%.

| Override rate | True engine failure rate | Rate you measure | Understated by |
| --- | --- | --- | --- |
| 5% | 6.75% | 5.00% | 1.35x |
| 10% | 8.50% | 5.00% | 1.70x |
| 15% | 10.25% | 5.00% | 2.05x |
| 25% | 13.75% | 5.00% | 2.75x |
| 40% | 19.00% | 5.00% | 3.80x |

Read the last column as the size of the illusion. The measured number never moves, because it is always the easy-case rate, while the true number climbs with the share of hard cases being removed.

The conclusion is uncomfortable and worth stating directly: the better your dispatchers are, the better your engine looks and the less it learns. Skilled intervention protects the operation and corrupts the evaluation at the same time. This also collides with governance, since [human oversight of automated allocation](https://locus.sh/blogs/agentic-ai-logistics-autonomous-execution-guide/)
 is increasingly mandated rather than optional in European deployments. The answer is not less oversight. It is recording what the engine proposed before the override, so the counterfactual survives the correction.

## Five questions to ask about a dispatch learning claim

**Which model improves, and on what data?** Ask whether the learning applies to travel and service time prediction, to allocation policy, or to both. The first is straightforward; the second requires labeled alternatives, so ask where those come from.

**Are rejected plans retained?** If the system stores only the executed plan, no allocation learning is possible in principle. This is a yes or no question about the data model, not a roadmap question.

**Is prediction error reported separately from plan adherence?** These get conflated constantly. Adherence measures whether execution followed the plan; prediction error measures whether the plan rested on accurate estimates. A plan can be followed perfectly and still have been built on a wrong travel time.

| Also Read: TMS Analytics: The Metrics That Drive Logistics ROI |
| --- |

**How are overrides captured?** Ask whether the pre-override plan, the reason and the outcome are all retained. Most systems keep the final state, which destroys the most valuable comparison in the operation.

**What is the arc and stop coverage of the learned model?** A travel-time model trained only on the corridors your current policy favors will keep favoring them. Coverage is the diagnostic that catches a self-reinforcing plan.

## What this looks like in enterprise deployments

A Fortune 50 parcel operation running [centralized dispatch](https://locus.sh/case-studies/fortune-50-parcel-centralized-dispatch/)
 across a 120-country network and 51 sites lifted weekly execution adherence from 75% to 92% and surfaced more than $14 million of unused capacity, including $565,000 at a single site. Both numbers are instructive. Adherence is always available, because plan against actual needs no counterfactual. And the unused capacity was found by comparing planned against feasible, not by mining history for a better policy. The measurement that moved the operation was one the data could actually support.

A global FMCG network running [logistics automation](https://locus.sh/case-studies/global-fmcg-logistics-automation/)
 across 10 Asian countries, 1,000-plus distributors and 5,000-plus riders reports 12,000-plus trips saved a month across 1.8 million retail outlets. A saving is a counterfactual claim by construction, since saved means relative to something that did not happen. That is why the baseline has to be fixed before the change and held afterwards. Operations that reconstruct it later find the comparison has moved underneath them, and a number nobody can reproduce is worth less than a smaller one anybody can.

## Four mistakes operations make about dispatch learning

**Treating data volume as the constraint.** More history improves estimates of what you observed and does nothing for what you did not. No accumulation rate turns an unobserved alternative into a labeled one, so teams that answer weak allocation learning by extending the retention window are solving the wrong problem.

**Overwriting the machine plan on override.** A schema decision made once, usually by someone optimizing for storage, permanently removes the ability to evaluate allocation quality. It is the most common and most expensive instance of this problem.

**Reading plan adherence as engine quality.** High adherence means execution followed the plan. It says nothing about whether the plan was good, and a poor plan followed faithfully produces excellent adherence.

**Assuming exploration is a luxury.** Without deliberate deviation an allocation policy can only confirm itself. A small, low-consequence exploration budget is the price of knowing whether it is right, and skipping it leaves the operation confident rather than correct.

## How Locus approaches decision records and learning

Locus, the world’s first Decision-Intelligent, Agentic TMS, runs a continuous Sense-Decide-Execute-Learn loop across more than 12 million automated decisions a day, and the learning half of that loop depends on what the platform records at decision time rather than on volume alone.

Explainability and Traceability capture the trigger, context, reasoning, action and outcome for each decision. The reasoning element is the one that matters here, because it retains what was considered and why one option won. That is a decision record rather than an event log, and it is the substrate a counterfactual comparison needs. Human Review preserves the machine proposal alongside the human decision, so an override survives as a pair rather than collapsing into a final state. Evaluation and the Execution Sandbox allow a policy change to be assessed before it reaches live plans, the controlled form of the exploration problem above.

Because the [route planning system](https://locus.sh/route-planning-system/)
 generates dispatch-ready plans in roughly two minutes against more than 250 real-world operating constraints, re-planning is cheap enough to be a measurement instrument in its own right. A plan recomputed under a changed assumption and compared to what ran produces a like-for-like counterfactual on the model even where the world supplies none.

Two boundaries belong here. Locus does not claim to recover outcomes for plans that were never executed, because nothing can. What the platform provides is the record that makes the comparison possible where it is, and the governance to run a deliberate alternative safely where it is not. And the instrumentation only pays off if the operation uses it. An override reason code is a process commitment from dispatchers rather than a feature toggle, and no platform supplies that on your behalf.

Locus supports more than 360 enterprise customers across 30-plus countries, with over 1.5 billion deliveries optimized, more than $320 million in documented client logistics savings and 99.99% uptime. It has been [recognized by Gartner for seven consecutive years](https://locus.sh/analyst-recognition/)
, featured in the 2026 Hype Cycle for Supply Chain Execution and Logistics Technologies, named a Leader in TMS by QKS Group (SPARK Matrix), and ranked #1 in Route Planning on G2’s 2026 Best Software Awards.

In October 2025, Ingka Investments, the investment arm of Ingka Group, the world’s largest IKEA retailer, acquired Locus. Locus continues to operate independently.

| Also Read: What Should a CXO Consider When Evaluating a Modern TMS |
| --- |

So can AI-driven dispatch learn a better allocation policy from your history? Partly, and the part matters. Travel time, service duration and address accuracy are observed on every execution regardless of what you decided, so those models genuinely improve and year-two plan quality genuinely beats year one. Allocation policy is different, because each cycle produces one executed plan and zero measured alternatives, and 365,000 plans over five years still contains zero observations of any alternative to any of them. Dispatcher overrides make it harder rather than easier: at a 15% override rate concentrated on hard cases, an engine failing on 40% of hard plans and 5% of easy ones measures at 5.0% against a true 10.25%, understating itself 2.05x, which means skilled intervention protects the operation while corrupting the evaluation. Closing that loop takes instrumentation rather than patience, and Locus supports it by recording the trigger, context, reasoning, action and outcome of each decision through Explainability and Traceability, preserving the machine proposal alongside the human one through Human Review, and testing policy changes in the Execution Sandbox before they reach live plans. [Request a Locus dispatch assessment](https://locus.sh/schedule-demo/)
 to review what your current system records at decision time.

## Frequently Asked Questions

**Does AI-driven dispatch actually learn from historical data?** Yes, for some models. Travel time on traversed arcs, service duration at visited stops and geocode accuracy at delivered addresses are observed on every execution regardless of the allocation decision, so they accumulate real labels and improve. Allocation policy quality is a different question, because it depends on alternatives that were never run.

**What is the counterfactual problem in dispatch?** Each dispatch cycle produces one executed plan and no measured outcome for any alternative. To know whether the engine allocated well you would need the realized cost of a plan it did not run, and that observation does not exist. The data is censored by construction rather than incomplete by accident.

**Why does more data not solve it?** Because sample size is not the constraint. Five years at 200 plans a day is 365,000 executed plans and still zero observations of any alternative. Accumulation sharpens estimates of what you observed and leaves what you did not observe exactly as unmeasured as it was on day one.

**How do dispatcher overrides distort performance measurement?** Overrides concentrate on hard cases, so the engine is measured on the easy remainder. With overrides on 15% of plans, an engine failing on 40% of hard plans and 5% of easy ones has a true failure rate of 10.25% but measures at 5.0%, understating itself 2.05x. At a 40% override rate the understatement reaches 3.80x.

**Should we reduce human oversight to improve learning?** No. Oversight protects the operation and in some jurisdictions is a regulatory requirement rather than a choice. The fix is to retain the machine proposal alongside the override and a reason code, so the comparison survives the correction instead of being overwritten by it.

**What is the cheapest way to start generating the missing labels?** Stop overwriting rejected plans. Retaining the pre-override plan, the reason and the outcome is a schema change rather than a project, and it converts an existing daily process into labeled training data. After that, reserving about 2% of low-consequence plans for a deliberate alternative yields roughly 1,460 labeled comparisons a year at 200 plans a day.

**How does arc coverage create a blind spot?** Travel-time models only learn about routes the policy chooses. If 30% of network arcs are traversed in a month, 70% carry no fresh timing, and any arc avoided because of a stale estimate stays stale because it stays avoided. Reporting coverage alongside accuracy is what exposes the loop.

MEET THE AUTHOR

Aseem Sinha

Vice President - Marketing

Aseem, leads Marketing at Locus. He has more than two decades of experience in executing global brand, product, and growth marketing strategies across the US, Europe, SEA, MEA, and India.

### Related Tags:

[https://locus.sh/blogs/ai-driven-dispatch-eu-ai-act-high-risk-2026/](https://locus.sh/blogs/ai-driven-dispatch-eu-ai-act-high-risk-2026/)
#### [General](https://locus.sh/blogs/category/general/)

## [AI-Driven Dispatch and the EU AI Act: What Your System Allocates on Decides its Risk Class](https://locus.sh/blogs/ai-driven-dispatch-eu-ai-act-high-risk-2026/)

[Anas T](https://locus.sh/blogs/author/anas_locus/)

Sep 10, 2026

Whether AI-driven dispatch is a high-risk AI system under Annex III depends on what it allocates on. Constraint-based and behaviour-scored allocation differ.

[Read more](https://locus.sh/blogs/ai-driven-dispatch-eu-ai-act-high-risk-2026/)

[https://locus.sh/blogs/carrier-visibility-coverage-funnel-multi-carrier-2026/](https://locus.sh/blogs/carrier-visibility-coverage-funnel-multi-carrier-2026/)
#### [General](https://locus.sh/blogs/category/general/)

## [Your Carrier Visibility Coverage Number is One Factor of Four](https://locus.sh/blogs/carrier-visibility-coverage-funnel-multi-carrier-2026/)

[Ishan Bhattacharya](https://locus.sh/blogs/author/ishan_locus/)

Sep 10, 2026

Coverage is a product of four conditions, not one percentage. A 95% tracked rate can be 58% actionable, and the gap sits in the factor nobody reports.

[Read more](https://locus.sh/blogs/carrier-visibility-coverage-funnel-multi-carrier-2026/)

## AI-Driven Dispatch Learns From What You Did, Not From What You Should Have Done

- Share
- [Print](javascript:window.print())
- [Download](#)
- [Schedule a Demo](https://locus.sh/schedule-demo/)

### Is your team spending more time on fixing logistics plan than running the operation?

- Agentic transportation management from order intake to freight settlement
- Route optimization built on 250+ real-world constraints
- AI-driven dispatch with automatic execution handling

20%Cost Reduction

66%Faster Planning Cycles

[Schedule a demo](/schedule-demo/)

Insights Worth Your Time

#### [General](https://locus.sh/blogs/category/general/)

## [Locus 2026 US Consumer Survey: Generative AI isn’t Just Changing How Consumers Shop, it’s Breaking the Demand Patterns US Retail Was Built On](https://locus.sh/blogs/generative-ai-shopping-effect-retail-fulfillment-operations-locus-q2-2026-consumer-survey/)

[Ishan Bhattacharya](https://locus.sh/blogs/author/ishan_locus/)

May 29, 2026

#### [General](https://locus.sh/blogs/category/general/)

## [Embedded vs Bolted-On AI: The Architecture Question European Logistics Buyers Are Asking](https://locus.sh/blogs/embedded-vs-bolted-on-ai-european-logistics-platform-architecture-business-benefits/)

[Aseem Sinha](https://locus.sh/blogs/author/aseem_locus/)

May 21, 2026

#### [General](https://locus.sh/blogs/category/general/)

## [Hybrid Fleet Management: How Owned, 3PL, Gig, ICE, and EV Capacity Actually Operate at Most Enterprises](https://locus.sh/blogs/three-workforce-fleet-reality-owned-3pl-gig-drivers/)

[Aseem Sinha](https://locus.sh/blogs/author/aseem_locus/)

May 7, 2026

#### [General](https://locus.sh/blogs/category/general/)

## [US Returns Hit $850 Billion in 2025: Why US Retailers Are Restructuring Reverse Logistics in 2026](https://locus.sh/blogs/850-billion-us-returns-ai-routing-reverse-logistics-2026/)

[Ishan Bhattacharya](https://locus.sh/blogs/author/ishan_locus/)

May 7, 2026