How It Works

Bayesian Truth Serum

A clever way to get honest answers by asking people to predict what others will say.

The Big Idea

Regular surveys have a problem. People sometimes lie or give answers they think you want to hear. Bayesian Truth Serum fixes this by asking you two questions instead of one.

Your Answer

What do you actually think?

Your Prediction

What will other people choose? (not including you)

The magic is in how we score you. You're paired with another respondent, and your scores depend on how your answers and predictions compare to theirs.

How You Earn Points

Two ways to score, both rewarding honesty

Information Score

You earn points when your answer matches your peer's answer—especially if they didn't expect many people to give that answer. Honest answers tend to be "surprisingly common."

Example. If your peer predicted only 20% would say "Yes" but you both said "Yes," you get bonus points for the unexpected agreement.

Prediction Score

You get points for accurately predicting what your peer will answer. The closer your prediction, the better you score.

Example. If you predicted 60% chance of "Yes" and your peer said "Yes," you score better than if you had predicted 20%.

Your Total Score is the Information Score plus the Prediction Score.

Higher is better. The most honest and thoughtful respondents score highest.

Why Honesty Wins

Here's the clever part. Lying actually hurts your score.

→If you lie about your answer, you're less likely to match your peer, reducing your Information Score.
→If you lie about your prediction, you'll be less accurate about your peer's answer, reducing your Prediction Score.
→Honest people naturally have an edge because they know their own true answer is shared by others like them.

The math is designed so that your best strategy is always to tell the truth, both about what you think and what you predict others will say.

Technical Details

The mathematical formulas for the curious

References

Drazen Prelec

"A Bayesian Truth Serum for Subjective Data"

Science, Volume 306, Issue 5695, pages 462–466, 2004

The foundational paper that introduced the Bayesian Truth Serum mechanism.

View paper →

Jens Witkowski and David C. Parkes

"A Robust Bayesian Truth Serum for Small Populations"

Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, 2012

Introduced RBTS for binary signals with strict incentive compatibility for n ≥ 3. Used for Yes/No questions in this app.

View paper →

Goran Radanovic and Boi Faltings

"A Robust Bayesian Truth Serum for Non-Binary Signals"

Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, 2013

Extended RBTS to handle multiple choice options with robustness guarantees. Used for questions with 3+ options in this app.

View paper →