Ry4ns4n, a friend and lecturer at Universitas Gadjah Mada, created and posted the infographic above a few months ago, which sets out some fascinating information on risk tolerance in Indonesia in a really readable way (if you happen to be able to read Indonesian).
The infographic above claims that 39% of men and 46% of women are classed as risk averse, which is categorised as people that would choose IDR 800,000 (guaranteed) over gambling between receiving IDR 800,000 and IDR 1,600,000.
This type of question is common in behavioural economics and studies of risk tolerance, but usually it’s of the form: which would you choose, a box with USD 10,000, or a box with a 10% chance of USD 100,000 and a 90% chance of nothing. In this sort of example, the expected value of the boxes is the same, but people often answer differently depending on their risk tolerance. In my experience polling people (which I’ve done rather a lot, because I have run quite a few trainings on this subject), the vast majority of people will take the guaranteed money, trading off the chance of more money for certainty.
In my example, you’re choosing between a sure thing, and something with potential upside and downside. In Ry4ns4n’s example, in fact, there’s no downside. You either receive IDR 800,000, or a chance at more with no downside. Who would take such an offer?
When I saw the infographic, I was convinced Ry4ns4n had made a mistake in interpreting the data, or the question. First off, I thought, there's no way such a large proportion of Indonesians would have answered that question that way? Then, secondly, why would you even put together a survey and ask this question that, surely, everyone would answer one way.
I pressed him, “surely you’ve got this wrong. The question makes no sense.” But he was firm that he was accurately interpreting and representing the data.
Still disbelieving, I went to the source, the fourth Indonesian Family Life Survey (IFLS), put together by the RAND Corporation.
If you’ve never seen the IFLS data set and you’re interested in, well, really, any sort of data driven analysis in Indonesia, you really, really should check it out. Respondents answer literally thousands of questions about everything from how many children they have, to how often they work, whether they can identify misspelled words, complete maths or logic problems, their incomes and expenditure, and—of key interest to me—risk tolerance; among many other topics. My guess would be, almost no matter what your field, there are probably a few questions relating to it in the IFLS.
The relevant question is question SI01 from Book 3A, and reads as follows:
Indonesian | English |
---|---|
Pertanyaan Misalkan Ibu/Bapak/Sdr ditawari dua pilihan untuk memperoleh pendapatan. Pada pilihan pertama, Ibu/Bapak/Sdr dijamin akan mendapatkan Rp 800 ribu per bulan. Pada pilihan kedua, Ibu/Bapak/Sdr dijamin akan mendapatkan Rp 1,6 juta per bulan atau Rp 800 ribu per bulan, masing-masing dengan kesempatan yang sama. Pilihan mana yang Ibu/Bapak/Sdr akan ambil? Jawaban 1. Rp 800 ribu per bulan 2. Rp 1,6 juta atau Rp 800 ribu per bulan 8. TIDAK TAHU |
Question Suppose you are offered two ways to earn some money. With option 1, you are guaranteed Rp 800 thousand per month. With option 2, you have an equal chance of ither (sic) the same income, Rp 800 thousand per month, or, if you are lucky, Rp 1.6 million. per month, which is more. Which option will you choose? Answers 1. Rp 800 thousand per month 2. Rp 1.6 million or Rp 800 thousand per month 8. DON’T KNOW |
Note: The wording above is directly from the IFLS4 surveys. The translation isn’t really one I would have chosen… If you want to see the full list of questions, check them out here.
So, yes, it looks like, in fact, the question is pretty much as Ry4ns4n represented it. And it looks like the RAND Corporation really intended it this way because the follow-up question if you select answer 1 is as follows:
Indonesian | English |
---|---|
Pertanyaan Apakah Ibu/Bapak/Sdr yakin? Pada pilihan pertama Ibu/Bapak/Sdr akan selalu mendapatkan Rp 800 ribu. Pada pilihan kedua Ibu/Bapak/Sdr akan mendapatkan setidaknya Rp 800 ribu per bulan dan mungkin akan mendapat Rp 1,6 juta per bulan. Jawaban 1. Tetap memilih pilihan 1 SI11 2. Berubah ke pilihan 2 8. TIDAK TAHU |
Question
Are you sure? In option 2 you will get at least Rp 800 thousand per month and you may get Rp 1.6 million per month. In option 1 you will always get Rp 800 thousand per month. Answers 1. Still picks option 1 2. Switches to option 2 8. DON’T KNOW |
I was still very confused at this point. The question seemed pointless to me, so I went in and looked at the results.
Sure enough, as Ry4ns4n already told me, the results were startling. When offered a choice between IDR 800,000 and an equal chance of IDR 800,000 and IDR 1.6 million, the results broke down like this:
So, from 29,054 respondents, 45%, or 13,079 respondents chose the guaranteed IDR 800,000 over the option with a chance of more and no chance of less.
Then, once they are asked “are you sure” and have it further explained to them, the results break down like this:
So, when they have the question further explained to them, only 91% of people that chose the worse option chose to stick with it.
So, out of the whole sample, even after being challenged, 12,043 people, or 41% of the sampled population would choose IDR 800,000 over equal chances at IDR 800,000 and IDR 1.6 million.
This is astounding
I really find this result astounding... I mean, pretty much all economists believe that agents act to maximise their utility or, at least, their perceived utility. Money is definitely not the only contributor to utility, but for small amounts of money, it's not very controversial to say that, all things being equal, almost anyone would prefer more to less.
I like to think that I have a broader-than-average understanding of what maximising utility means, and I like to think I'm pretty good at identifying where different groups might react dramatically differently to the same stimuli. But, if you had asked me to guess what proportion of people would choose option 1, I would probably have been willing to bet quite a lot of money that it would be below 10%. Under almost no circumstances would I think it would exceed 20%.
41%? Well, that's so far past my reasonable expectations that I start to question the the rationality assumption, which rests right at the centre of pretty much all mainstream economics.
So, what’s going on?
The short answer is: I don’t know. I’ll hazard a few guesses, but these are very preliminary. More notes to myself on further avenues to look down, and hypotheses to test when I get some time to dig through the data a bit more. In the meantime, if anyone wants to challenge/debunk my theories, or has some alternative theories on what’s going on and wants to run some numbers and/or do their own write up, I’d be happy to link/cross-post/host a guest-post.
My first theory is that there will have been a certain proportion of people that just won’t have understood the question due to low mathematical literacy. As I noted in a previous post, uncertainty confuses people, and a lot of people won’t have had practice at breaking down a problem like this into a mathematical framework*.
My guess is that this proportion probably isn’t that large, because later there’s a similar question with the pay-offs reversed, where the respondents are asked to choose between IDR 4 million and a IDR 4 million with a chance of IDR 2 million, and 90% of people answer the “right” way there. Ry4ns4n’s infographic provides some information on propensity to be what he calls “risk averse” with level of education, but the results seem a bit up and down… There are some maths and logic questions early in the survey, you could perhaps use those to try see whether the proportion of people that chose IDR 800,000 were over-represented in the population with the lowest mathematical literacy.
I would also guess that there would be a proportion of people were probably mentally exhausted by the length of the questionnaire, and were bamboozled by the gear shift from a series of questions about their personal plans for retirement, to a question that sounds like an IQ test. There’s a series of questions at the end of the section about how engaged and attentive the respondent was, but I haven’t gotten around to exporting the data on those ones, plus that’s going to be a very subjective call, and it applies to the whole book of questions, not just the section I’m looking at here. I suspect this is going to be tough to try and test.
Next, I think there will be a certain proportion of people that will think this sounds too much like gambling, which is moderately taboo in a reasonable proportion of Indonesia, and will reject it on that basis. This could explain some of the difference in people that are happy to remove uncertainty on the downside, but don’t want to take it on the upside. I haven’t looked enough through the survey to know if there are any questions on attitudes to gambling, but in their absence, maybe you could overlay the responses with self-reported levels of religiosity.
Finally, and I think, most importantly, I think we come down to the issue of trust. I wrote about this in the same blog post I linked above and also here. Most of the people that take this survey are relatively poor, and poor people have very low levels of trust with fancy-sounding schemes. You might think you’re just asking a simple question about risk tolerance, but a poor person might look at the offer and assume that you’re trying to cheat them somewhere. After all, they have been trained in the real world where, when someone offers you something for nothing, you’re best off walking away as soon as possible, because they're probably trying to cheat you. My guess would be that this is the biggest factor.
I’m very interested in why the RAND Corporation saw fit to include this question. This section of the survey is called “RISK AND TIME PREFERENCES.” I’m not sure that it’s accurate to say that answers to this question reveals anything about a respondent’s risk tolerance or time preference. I think it reveals more about mathematical literacy, tolerance for situations that sound like gambling, or trust in institutions than in someone’s tolerance for risk. After all, nothing is actually at risk in this question.
But then, I wonder if, maybe, that’s the point.
In this section, you want to get people to reveal their risk tolerances. It’s easier to get them to do that if they are answering the questions in the frame of mind where they understand the questions, are happy to weigh up options that sound like gambling, etc. etc. If you ask this question first, maybe it can act as a nice filter to get rid of the people that aren’t going to answer your questions in a way that is useful to you. That way, when you get into the questions about how they actually do want to bear risk, you’ve removed a whole lot of people that will introduce a lot of crazy noise into your dataset. Maybe...
As I noted above, I very much welcome any challenges or thoughts on my theories and questions above.
So what?
After I finally accepted that, yes, indeed, the question was correctly phrased, and that the answers appeared to be legitimate, I still boggled over this for quite some time. It took quite a while, but now I am at peace with it.
Whatever drove 41% of respondents to answer these questions the way they did, I think these results serve as a great reminder that you should never assume you know how any group of people thinks**. Always, always question your preconceptions and assumptions, and test them where possible, because people will surprise you.
* This reminds me of a story a teacher friend of mine told me once. She was teaching a maths class to a bunch of year 7s in a high school in Australia, and was describing how the uniform distribution of rolling one die gets turned into a peaked distribution by rolling two. She was unable to convince a good chunk of her class that a single dice had a uniform distribution until she sat with them, rolling a single die, building a histogram for 20 minutes. “But, it’s harder to roll a 6 than it is a 1, right?” This stuff is just not that intuitive unless you really think about it a lot…
** This will be the third and last time I will link to this post, but I did want to drop in this graph that I saw on Chris Blattman’s blog post from the World Bank’s 2015 World Development report.