It's probably not great to attack a book when I've barely started it, but early in Chapter 1 of Steven Pinker's "Rationality", he mentions an old chestnut of a riddle
On a field there is a patch of weeds. Every day the patch doubles in size. It takes 30 days for the patch to cover the whole field. How long did it take for the patch to cover half the field?and Pinker bemoans that most people won't get the correct answer (29 days, i.e. since it doubles daily, the day before the final day it was half the final size.)
Human intuition doesn't grasp exponential (geometric) growth, namely something that rises at a rising rate, proportional to how large it already is, such as compound interest, economic growth, and the spread of a contagious disease. People mistake it for steady creep or slight acceleration, and their imaginations don't keep up with the relentless doubling.But what a fantastical setup that riddle is! Like any physical model would show us that no patch of weeds on earth could have that kind of behavior "steadily" over 30 days. To show that to myself, I hacked my version of Conway's Game of Life to be even simpler : every alive cell lives on, and every dead cell with at least one alive neighbor is born. The result is visually boring - a square that grows from the middle of the screen. And checking the population numbers, they are far from doubling. The rate that the square can grow is clearly bounded by its boundary, the 2D "surface area" where it has new fertile territory to move into, and so there's no way its actual area could keep doubling. And similarly, I can't think of a mechanism and environment that would support much of anything from having consistent doubling behavior for 30 days!
I find these thought experiments infuriating when they are used as examples of people's "irrationality". It's akin to economists thinking people are irrational for preferring receiving ten dollars now vs thirty dollars a year from now. In an uncertain world, any real world test subject is absolutely correct to be suspicious of a test program reliably running over the course of a year (especially when its business model seems to have big deal of just giving away money!)
I used to think of these as "casino-ish" problems- like, they are customized to prey on human's response at this attractive edge of artifice. But I guess I'd say they're "hothouse gullibility" thought experiments - they take for granted that OF COURSE the research is trustworthy, or that a patch of weeds that doubles every day for 30 days is a meaningful prototype to ponder. They are merely interrogating how well subjects can navigate a completely artificial environment of simplifying assumptions.
Update... later in explaining why people make this kind of error he does say
we might point to the ephemerality of exponential processes in natural environments (prior to historical innovations like economic growth and compound interest). Things that can't go on forever don't, and organisms can multiply only to the point where they deplete, foul, or saturate their environments, bending the exponential curve into an S. This includes pandemics, which peter out once enough susceptible hosts in the herd are killed or develop immunity.So I think it still forces the question: how meaningful are these contrived examples in generating useful knowledge about the world?