[ PUBLICATION / EVALUATION ]
Predicting national survey results before they are published
An out-of-sample evaluation of Rebel Labs’ simulation system on 250 questions from nine national pollsters. Predicted response distributions overlapped with published results by 90.6% on average, within the variation observed when the same survey is fielded twice.
Overview
Survey research remains the standard for understanding how populations think and decide, but it is slow and expensive, and its results often arrive after the decisions they were meant to inform. This study measures how closely Rebel Labs’ simulation system can anticipate what a national survey will find, before the results exist.
We evaluated the system on 250 questions from nine national pollsters: Pew Research Center, Economist/YouGov, Ipsos, Gallup, KFF, Quinnipiac, Marist, the University of Michigan Surveys of Consumers and the New York Fed Survey of Consumer Expectations. Every question was fielded between July 20 and October 19, 2026, after the training cutoff of the underlying language model. The set spans seven domains: finance, social attitudes, politics, technology, consumer behavior, health and energy.
Results
90.6%
Distribution accuracy
Predicted response distributions overlapped with published results by 90.6% on average; the median question reached 92%.
88%
Decision-grade
Nearly nine in ten predictions fell within 16 points of the published distribution.
4.3 pts
Average error
Average error per response option, with a 0.95 correlation between predicted and published shares.
88–98%
Replication range
The same survey fielded twice overlaps with itself by roughly 88 to 98%. Rebel Labs’ predictions fall inside that range.
Distribution accuracy measures how much of the predicted answer distribution coincides with the published one. A poll finding 60% yes and 40% no, predicted at 55% and 45%, scores 95%.
Comparison with biography-prompted simulation
The prevailing approach to synthetic respondents gives a language model a demographic biography and asks it to answer as that individual, then aggregates many such answers into a distribution. We ran this approach on the same 250 questions with the same model, drawing 150 personas per question from Census microdata.
Metric
Rebel Labs
Biography-prompted
Distribution accuracy
90.6%
87.0%
Decision-grade
88%
70%
Top answer correct
80%
73%
Error per response option
4.3 pts
5.8 pts
Performance comparison
Distribution accuracy
Rebel Labs
90.6%
Biography-prompted
87%
Decision-grade
Rebel Labs
88%
Biography-prompted
70%
Top answer correct
Rebel Labs
80%
Biography-prompted
73%
Persona-based methods infer the population from individuals role-played one at a time. Rebel Labs models the population directly, grounded in the conditions respondents were living through when the survey was fielded. The gap was widest in energy, where views depend on conditions that change week to week.
Results by domain
Domain
Questions
Accuracy
Decision-grade
Top answer
Social attitudes
41
92.0%
93%
76%
Finance
55
91.5%
93%
82%
Politics
35
91.1%
89%
86%
Health
31
90.4%
84%
71%
Technology
35
90.0%
86%
80%
Consumer
32
89.0%
88%
81%
Energy
21
88.7%
76%
81%
Distribution accuracy by domain
Social attitudes
92.0%
Finance
91.5%
Politics
91.1%
Health
90.4%
Technology
90.0%
Consumer
89.0%
Energy
88.7%
Accuracy held across domains. Social attitudes and finance were strongest; energy was the most demanding.
Examples
Four questions, each typical of its domain, comparing the published result with Rebel Labs’ prediction.
Pew
Election system security
93.0%
Economist/YouGov
Outlook on quality of life
93.0%
Economist/YouGov
Trust in unemployment figures
92.0%
Quinnipiac
Trust in AI company leaders
89.5%
Method
For each question, the system receives the wording, response options, target population and fieldwork window, along with a structured summary of real-world conditions in the weeks before fieldwork: shifts in public attention, prices and spending relative to the prior year. It never sees the pollster, sponsor, survey mode or any result, and returns a full response distribution that is recorded before the published figures are consulted.
The evaluation was designed so that no result could leak into a prediction. Every question was fielded after the model’s training cutoff, and every input was published before fieldwork began. System design used a separate development set of 50 questions; the 250 reported here were held out. Every question is scored and every prediction is logged.