Why a good demonstration can stall
The federal government has invested in AI through strategies, research programs and operational initiatives. Moving from those investments to dependable daily use requires several organizations to make compatible decisions: the mission owner, data owner, acquisition team, security organization and operators.
GAO's review of AI-enabled weapon-system development identified challenges including usable data, testing and workforce expertise. Model performance, organizational readiness and integration all deserve examination when a pilot stalls. A demonstration under curated conditions resolves only part of the deployment question.
A model may perform well on an approved sample while the production application still lacks access to the necessary records. A program may have funding for experimentation but no organization willing to pay for hosting and support. Users may find the output accurate yet too slow to review during their actual work. Each is a different transition problem and requires a different remedy.
The useful management question is therefore specific: what has the pilot proved, and which assumptions still stand between that result and routine operation?
Put operational data in the plan early
Data problems can appear as a permissions issue, a quality issue or an integration issue. Treating all three as a need for more training data can waste time.
- Access: Can the authorized production user and service obtain the records they need, at the required classification and dissemination level?
- Quality: Are errors, missing values and inconsistent definitions understood well enough to evaluate their effect?
- Representativeness: Does the test set include the conditions and exceptions that operational users encounter?
- Continuity: Who owns the feed, how are changes announced and what happens when it stops?
The foundation includes pipelines, metadata, quality checks and governance. Those investments should proceed alongside model evaluation. A team need not finish every enterprise data project before testing a small use case, but it should avoid declaring success using a data arrangement that cannot be sustained.
For example, a manually assembled test collection may be sufficient to explore a concept. It is insufficient evidence for a service expected to reconcile newly arriving records every day. The transition plan needs to expose that difference and fund the work to close it.
Use acquisition flexibility with clear acceptance criteria
AI development often involves iteration as teams learn about their data and users. That does not make measurable performance requirements inappropriate or make every government acquisition process incompatible with AI.
A requirement such as 95 percent accuracy is incomplete without defining the task, sample, error costs and operating conditions. It may hide poor performance on uncommon but consequential cases. A stronger requirement combines operational outcomes with explicit evaluation measures, tolerable failure conditions and a process for assessing changes.
Review update, September 2026: GAO's 2026 acquisition review describes tradeoffs involving competition, data and intellectual property, and calls for systematic collection and sharing of acquisition lessons. It reinforces the value of retaining lessons from a pilot in the contract and transition record, rather than letting them disappear with the demonstration team.
Contract planning should address data rights, integration responsibilities, testing access, model changes and exit arrangements. Iteration is useful when the government can determine what changed and whether the result improved. An open-ended promise of continuous improvement is not an acceptance method.
Fund operation as well as development
Deployment begins a continuing obligation to monitor the service, maintain dependencies, manage access and respond to changing conditions. Some models can remain stable for long periods; others need frequent revision. Neither automatic updates nor a permanently frozen version should be the default without a reason.
Sustainment planning should cover:
- Hosting, licensing and the staff who operate the service.
- Data maintenance and the work needed to resolve broken integrations.
- Evaluation after changes to models, sources or mission requirements.
- Security monitoring, incident response and rollback.
- User training, support and a way to report disappointing or unsafe results.
An iterative development process can support this work. The funding, release authority and operational staffing still have to exist. A development pipeline does not create them by itself.
Five decisions before the next expansion
- Name the owner. Identify the organization responsible for the service after the pilot team leaves.
- Define the workflow. Involve the users who will act on the output and the people affected by errors.
- Test realistic conditions. Include imperfect data, exceptions and the expected review workload.
- Secure the transition. Identify funding, procurement actions, hosting and approval dependencies.
- Set a continuation threshold. Decide what evidence justifies scaling, redesigning or ending the effort.
These decisions make the next demonstration more informative. They also let a team stop an unsuitable approach early without treating every experiment that ends as a failure. The goal is a capability the organization can use, support and improve with a clear understanding of its limits.
Sources and further reading
- GAO: Developing and acquiring AI capabilities for weapon systems
- GAO: Lessons learned from federal AI acquisitions, 2026
Spartan X combines AI consulting with program execution to connect a promising use case to its data, acquisition and operating commitments. That connection is what turns a pilot decision into a supportable delivery plan.
