Artificial intelligence (AI) is supposed to make business smarter, faster and more efficient. But in 2025, it is just as often making things up. Hallucinations, the confident invention of facts, figures and sources, have become a headline risk.
A federal judge in Wyoming threatened to sanction lawyers who submitted AI-generated briefs laced with fictitious cases, while Butler Snow, a major firm, admitted in May that its attorneys relied on hallucinated citations.
What might look like quirky tech failures in consumer chat apps quickly turns into reputational and regulatory landmines when applied to banking, payments or compliance.
A Business Problem, Not a Bug
The industry once dismissed hallucinations as teething errors. Today, they are seen as structural. The Wall Street Journal reported that leading developers are training AI to say “I don’t know” rather than improvise, recognizing that probabilistic models will never be error free.
The Financial Times has also covered this tension, warning that the “hallucinations that haunt AI” reflect fundamental challenges in how language models reason and the difficulty of policing them.
MIT Sloan’s 2025 commentary warned that banning AI tools is not realistic. Instead, firms must define clear guardrails, designate where human review is mandatory, and train staff to treat AI output with skepticism. The key is building a culture of verification.
Hallucinations carry mounting cost. In March, Bloomberg flagged that Wall Street firms are warning investors of AI risks, including hallucinations, as financial models and compliance systems increasingly rely on generative tools.
In September 2025, OpenAI published “Why Language Models Hallucinate,” reframing hallucinations not as rare bugs but as systemic effects of how models are trained and validated. The paper argues that benchmarks often reward confident guesses over caution, pushing models to bluff when uncertain. PYMNTS covered the release, noting that enterprises must now treat hallucination risk as endemic rather than exceptional.
Financial Services Draw a Line
This year, FICO launched a foundation model tuned for financial services, embedding transparency, auditability and domain anchoring to reduce hallucinations in payments, credit and compliance workflows.
But FICO is no outlier. The Financial Times recently asked “AI: what will become of the truth?” reflecting growing concerns that generative systems are eroding clarity between fact and fiction.
Courts are also tightening the screws. Judges are ordering lawyers to disclose AI use and sanctioning those who do not, responding to hallucinated filings. As reported by Reuters, regulators are following. State attorneys general have begun applying consumer protection laws to AI misinformation in financial products.
In payments, hallucinations scale fast and hurt harder. A misinterpreted compliance rule or fabricated sanction list entry could freeze legitimate flows or, worse, permit restricted transfers. For a system handling billions of transfers daily, even a 1% hallucination rate yields thousands of errors, each with regulatory and reputational risk.
Innovation is also threatened. Firms experimenting with AI in onboarding, disputes or cross-border compliance are hitting a ceiling. Without strong controls, hallucination risk stalls deployment. As one MIT Sloan researcher put it this year, “You cannot scale what you cannot trust.” For a sector built on instant settlement and real-time fraud detection, there is little room for improvisation.
From Perfection to Predictability
No one in the research community expects hallucinations to vanish. OpenAI’s September 2025 paper argued that hallucinations are systemic byproducts of how models are trained and evaluated.
The emphasis now is on predictability. Banks are piloting hallucination dashboards that record error rates and uncertainty signals. Payment networks are forcing models to cite verified compliance sources before approving transactions. Vendors like FICO are rolling out domain-specific tools to reduce hallucination risk.
In the payments realm, signs of this shift are already visible. Lloyds Bank and Coinbase, for example, recently reported by PYMNTS, increased confidence in their hallucination guardrails after deploying safer generative AI systems. AWS is rolling out automated reasoning safeguards to help financial and compliance systems catch hallucinated outputs before they reach customers.
Even the insurance industry is responding: Insurers have begun offering policies to cover AI-related errors, including hallucinated outputs, highlighting how serious the risk has become.
Source: https://www.pymnts.com/
