Your AI Has Opinions About Money
What are the practical implications of this paper for anyone developing products on top of LLMs and AI agents (https://www.nber.org/papers/w34745)? Do generative AI models, particularly large language models (LLMs), exhibit systematic behavioral biases in economic and financial decisions? If so, how can these biases be mitigated? Drawing on the cognitive psychology and experimental economics literatures, we conduct the most comprehensive set of experiments to date—originally designed to document human biases—on prominent LLM families across model versions and scales. We document systematic patterns in LLM behavior. In preference-based tasks, responses become more human-like as models become more advanced or larger, while in belief-based tasks, advanced large-scale models frequently generate rational responses. Prompting LLMs to make rational decisions reduces biases.
Asked by someone building something, who just realized they should probably understand the weird inherited quirks of the thing they're building on top of.
Here's the uncomfortable finding: the more advanced and capable an LLM becomes, the more it thinks like us about money.
Researchers gave LLMs the same trick questions behavioral economists have been using on humans for decades—gambler's fallacy, conjunction fallacy, the whole catalog of cognitive traps that make us lousy intuitive statisticians. Then they ran the same models through preference experiments, the kinds of scenarios where humans reliably choose against their own interests because loss aversion or status quo bias or some other ghost in our evolutionary firmware kicks in.
The split they found is strange. On belief-based questions—the ones where there's a correct answer and you either reason your way to it or you don't—advanced models mostly get it right. GPT-4, Claude 3 Opus, and Gemini 1.5 Pro produced largely rational answers to questions about probability and statistical inference. Smaller models and older versions stumbled into the same traps humans do, but scale and advancement seem to buy you out of those particular failures.
But preference-based questions work in reverse. The more advanced and larger the model, the more human-like (which is to say, the more irrational) its preferences become. The models aren't failing to compute; they're successfully imitating the biased patterns in their training data. They learned from us, and we are not rational agents.
If you're building on LLMs, this is where it gets uncomfortable. When your system needs to make probabilistic judgments or statistical inferences, your bigger models are probably fine—they've learned enough to escape some of our cognitive shortcuts. But when your system is making decisions that involve tradeoffs, risk, or anything that looks like preferences, you may have accidentally inherited human irrationality at scale. That AI financial advisor you're building might exhibit loss aversion. Your agent might discount future rewards too steeply, preferring a small reward now over a larger one later.
You didn't program these behaviors. You inherited them, the same way you inherited your grandmother's china and your father's bad knee.
The fix turns out to be weirdly simple, at least according to this research: just ask the model to be rational. A brief role-priming instruction—something like "think of yourself as a rational investor who makes decisions using the Expected Utility framework"—significantly reduces biased responses on both preference and belief tasks. More elaborate strategies didn't help; piling on additional bias-reducing information either made no difference or made things worse. The models apparently know what rational behavior looks like. They're just defaulting to human patterns unless you specifically ask them not to.
We've built systems that have absorbed our collective written wisdom on probability theory, game theory, and decision science. They can explain expected utility maximization perfectly. They can describe every cognitive bias in the catalog. And yet, when you just ask them to make a decision, they fall into the same traps we do. They learned rationality from our textbooks and irrationality from everything else we wrote.