Psychology Deception - Acceptable In The 80s

Apparently some 30-50% of psychology experiments published in top journals use deception (Hertwig and Ortmann, 2001). Why?

One reason for deceiving subjects is that it enables experimenters to create interesting situations. For example, we might want to see how people react in an emergency. Another reason is that it allows experimenters to hide the real purpose of the experiment from subjects. For example, we might want to stop people just giving the politically correct answers instead of what they really think (Nick Wilkinson, 2008).

However, the use of deception is frowned upon by economists.

The main problem is that people aren't stupid. Word gets round. Only the naive would enter a psychology experiment without the expectation of deception on the part of the experimenter. This has knock-on effects on behaviour. If you suspect you're being deceived you may just behave differently thus defeating the whole point of the experiment in the first place.

Thus there are few examples of deception in the world of experimental economics. This should mean that our results stay reliable, even if in the short term we are more limited in what we can do. In an ideal world deception would never ever be used in any experiment, but sadly there is little incentive for everyone to act for the greater good (cf. the free-rider problem).

Fully aware of the irony, I am going to end this post by saying that guruhogg has never knowingly used deception.

Recommended listening:
Acceptable In The 80s by Calvin Harris

Incentives - Ante Up

In a recent blog I outlined a common experiment called the Ultimatum Game and asked people what they would do. Much thanks to one reader who hit on a big issue in behavioural economics, that of incentivisation.

She pointed out that the money was purely theoretical (sadly guruhogg has no real money to offer). Therefore, how can we be sure that the behaviour we observe is what would really happen? Individuals may say they will do one thing, but if the decision had real consequences they might act differently.

Real economics experiments are very careful to make sure that subjects have an incentive to answer honestly. This is done by making real money ride on the decisions people make. In the Ultimatum Game example there would be an incentive because of the £10 riding on it.

Experiments which do not involve adequate incentivisation tend not to be taken seriously in the academic world. However, there is a debate over what constitutes an adequate incentive. Some economists believe that experiments which offer only ten or twenty pounds aren't a good proxy for behaviour in real life, where important decisions can involve thousands of pounds. In response, it's argued that the few economists that have somehow found enough money to offer huge incentives tend not to find systematically different behaviour.

Anyway, I readily admit that the experiments on guruhogg do not offer adequate incentivisation, but that doesn't matter as I'm not collecting results. Incentives will have to wait until guruhogg has found some way of making money!

Recommended listening:
Ante Up by M.O.P.

Opposition - The Imperial March

"There are ongoing disputes about what economics should learn from experimental results, about whether (or in what sense) economic theory can be tested in laboratory experiments, and about how far traditional theory needs to be adapted in the light of experimental results."

- Bardsley et al. (2010, p.1)

The Nottingham lab - http://www.nottingham.ac.uk/~lezorsee/img/lab.jpg

I've recently being doing some research into common objections to the use of experiments in economics. Here follows a brief summary...

Bardsley et al. (2010, p.9) explain a paper by Ken Binmore (1999): "Economic theory, he argues, can reasonably be expected to apply only under particular conditions..." (i.e. putting people in a special room and telling them they can't communicate with anyone else is unrealistic).

Bardsley et al. (2010, p.10) also summarise a paper by Steven Levitt (Freakanomics co-author) and John List (2007): "Levitt and List point out that, in many of the environments studied by economists, decision makers are not a representative sample of the population." (i.e. stock brokers are not stock brokers because they got chosen at random, but because they want to be stock brokers and are good at it). A potential problem as economists tend to select their subjects at random, rather than specially get in stock brokers etc.

http://bellarmine2.lmu.edu/econlab/inside.jpg

Vernon Smith (Nobel prize winner) notes a different issue with experiments. By their very nature experiments have underlying assumptions. These assumptions can always be contested (not always legitimately). Thus any experimental conclusion can be contested. This "denies the possibility of direct falsification of any specific testable implication of a theory" (1994, p.127).

On a related note, according to Alvin Roth (1988, p.1023) "the major pitfall to be aware of here is that... there is room for an experimenter's prior beliefs about the likely outcome of the experiment to influence the outcome, through these design decisions."

Roth concludes that "The danger is of inadvertently reading experimental evidence as supporting an overly general conclusion on observations made in special cases." (p.1023)

Despite all this, many of the above objections can be answered (another blog etc etc). Therefore, I still believe that experiments can shed valuable light upon economic theories. And by dealing with the citicisms experimental economics improves in both accuracy and validity. Obviously there are flaws (as with any method of study), but let's not throw the baby out with the over-used metaphor.


Recommended listening:
The Imperial March by John Williams

Key references:

Bardsley, N., R. Cubitt, G. Loomes, P. Moffatt, C. Starmer and R. Sugden (2010)  Experimental Economics: Rethinking the Rules  

Smith, V. L. (1994) "Economics in the Laboratory" Journal of Economic Perspectives, 8, 113-31 

Roth, A. (1988)  "Laboratory Experimentation in Economics: A Methodological Overview", Economic Journal,  974-1031.

Mood Matters – Feeling Good

http://www.techno-lovers.com/wp-content/uploads/2012/05/Samsung-Patented-Emoticon.gif

A recent Facebook conversation got me thinking.

My friend had just done the Common Ratio Effect experiment (see previous blog: Are You Consistent? - Mad World), and was pleased that she had been consistent. However in relation to her answers, she noted that:

“lol... depends on circumstances...”

Which is true. Our answers one minute may vary wildly from the next minute. One day we may be feeling positive and more prone to taking risks, another day we may be more conservative and risk-averse. 

So, therefore, aren’t experiments a load of rubbish? Surely they can’t capture human behaviour accurately, because human behaviour is inherently dependent on our mood?

However, economists solve this problem by repeating the experiments on lots and lots of people (and use different experiments to see if phenomena are ‘robust’). Thus any mood variability should balance out over large populations. And remember, economists test to see if an aspect of behaviour can be shown to be systematic among us humans which does not usually require literally everyone to exhibit it. Anomalies are allowed. It does not matter if you answer differently according to how you feel because an experiment will never rely on your answers alone.

There are other criticisms of experiments (which will have to wait for another blog), but the “it all depends on how I feel” objection, while being absolutely correct, does not actually challenge the validity of experiments.

Recommended listening:

Feeling Good by Nina Simone