So, when they have the question further explained to them, only 91% of people that chose the worse option chose to stick with it.
So, out of the whole sample, even after being challenged, 12,043 people, or 41% of the sampled population would choose IDR 800,000 over equal chances at IDR 800,000 and IDR 1.6 million.
This is astounding
I really find this result astounding... I mean, pretty much all economists believe that agents act to maximise their utility or, at least, their perceived utility. Money is definitely not the only contributor to utility, but for small amounts of money, it's not very controversial to say that, all things being equal, almost anyone would prefer more to less.
I like to think that I have a broader-than-average understanding of what maximising utility means, and I like to think I'm pretty good at identifying where different groups might react dramatically differently to the same stimuli. But, if you had asked me to guess what proportion of people would choose option 1, I would probably have been willing to bet quite a lot of money that it would be below 10%. Under almost no circumstances would I think it would exceed 20%.
41%? Well, that's so far past my reasonable expectations that I start to question the the rationality assumption, which rests right at the centre of pretty much all mainstream economics.
So, what’s going on?
The short answer is: I don’t know. I’ll hazard a few guesses, but these are very preliminary. More notes to myself on further avenues to look down, and hypotheses to test when I get some time to dig through the data a bit more. In the meantime, if anyone wants to challenge/debunk my theories, or has some alternative theories on what’s going on and wants to run some numbers and/or do their own write up, I’d be happy to link/cross-post/host a guest-post.
My first theory is that there will have been a certain proportion of people that just won’t have understood the question due to low mathematical literacy. As I noted in a previous post, uncertainty confuses people, and a lot of people won’t have had practice at breaking down a problem like this into a mathematical framework*.
My guess is that this proportion probably isn’t that large, because later there’s a similar question with the pay-offs reversed, where the respondents are asked to choose between IDR 4 million and a IDR 4 million with a chance of IDR 2 million, and 90% of people answer the “right” way there. Ry4ns4n’s infographic provides some information on propensity to be what he calls “risk averse” with level of education, but the results seem a bit up and down… There are some maths and logic questions early in the survey, you could perhaps use those to try see whether the proportion of people that chose IDR 800,000 were over-represented in the population with the lowest mathematical literacy.
I would also guess that there would be a proportion of people were probably mentally exhausted by the length of the questionnaire, and were bamboozled by the gear shift from a series of questions about their personal plans for retirement, to a question that sounds like an IQ test. There’s a series of questions at the end of the section about how engaged and attentive the respondent was, but I haven’t gotten around to exporting the data on those ones, plus that’s going to be a very subjective call, and it applies to the whole book of questions, not just the section I’m looking at here. I suspect this is going to be tough to try and test.
Next, I think there will be a certain proportion of people that will think this sounds too much like gambling, which is moderately taboo in a reasonable proportion of Indonesia, and will reject it on that basis. This could explain some of the difference in people that are happy to remove uncertainty on the downside, but don’t want to take it on the upside. I haven’t looked enough through the survey to know if there are any questions on attitudes to gambling, but in their absence, maybe you could overlay the responses with self-reported levels of religiosity.
Finally, and I think, most importantly, I think we come down to the issue of trust. I wrote about this in the same blog post I linked above and also here. Most of the people that take this survey are relatively poor, and poor people have very low levels of trust with fancy-sounding schemes. You might think you’re just asking a simple question about risk tolerance, but a poor person might look at the offer and assume that you’re trying to cheat them somewhere. After all, they have been trained in the real world where, when someone offers you something for nothing, you’re best off walking away as soon as possible, because they're probably trying to cheat you. My guess would be that this is the biggest factor.
I’m very interested in why the RAND Corporation saw fit to include this question. This section of the survey is called “RISK AND TIME PREFERENCES.” I’m not sure that it’s accurate to say that answers to this question reveals anything about a respondent’s risk tolerance or time preference. I think it reveals more about mathematical literacy, tolerance for situations that sound like gambling, or trust in institutions than in someone’s tolerance for risk. After all, nothing is actually at risk in this question.
But then, I wonder if, maybe, that’s the point.
In this section, you want to get people to reveal their risk tolerances. It’s easier to get them to do that if they are answering the questions in the frame of mind where they understand the questions, are happy to weigh up options that sound like gambling, etc. etc. If you ask this question first, maybe it can act as a nice filter to get rid of the people that aren’t going to answer your questions in a way that is useful to you. That way, when you get into the questions about how they actually do want to bear risk, you’ve removed a whole lot of people that will introduce a lot of crazy noise into your dataset. Maybe...
As I noted above, I very much welcome any challenges or thoughts on my theories and questions above.
So what?
After I finally accepted that, yes, indeed, the question was correctly phrased, and that the answers appeared to be legitimate, I still boggled over this for quite some time. It took quite a while, but now I am at peace with it.
Whatever drove 41% of respondents to answer these questions the way they did, I think these results serve as a great reminder that you should never assume you know how any group of people thinks**. Always, always question your preconceptions and assumptions, and test them where possible, because people will surprise you.