Opposition - The Imperial March

"There are ongoing disputes about what economics should learn from experimental results, about whether (or in what sense) economic theory can be tested in laboratory experiments, and about how far traditional theory needs to be adapted in the light of experimental results."

- Bardsley et al. (2010, p.1)

The Nottingham lab - http://www.nottingham.ac.uk/~lezorsee/img/lab.jpg

I've recently being doing some research into common objections to the use of experiments in economics. Here follows a brief summary...

Bardsley et al. (2010, p.9) explain a paper by Ken Binmore (1999): "Economic theory, he argues, can reasonably be expected to apply only under particular conditions..." (i.e. putting people in a special room and telling them they can't communicate with anyone else is unrealistic).

Bardsley et al. (2010, p.10) also summarise a paper by Steven Levitt (Freakanomics co-author) and John List (2007): "Levitt and List point out that, in many of the environments studied by economists, decision makers are not a representative sample of the population." (i.e. stock brokers are not stock brokers because they got chosen at random, but because they want to be stock brokers and are good at it). A potential problem as economists tend to select their subjects at random, rather than specially get in stock brokers etc.

http://bellarmine2.lmu.edu/econlab/inside.jpg

Vernon Smith (Nobel prize winner) notes a different issue with experiments. By their very nature experiments have underlying assumptions. These assumptions can always be contested (not always legitimately). Thus any experimental conclusion can be contested. This "denies the possibility of direct falsification of any specific testable implication of a theory" (1994, p.127).

On a related note, according to Alvin Roth (1988, p.1023) "the major pitfall to be aware of here is that... there is room for an experimenter's prior beliefs about the likely outcome of the experiment to influence the outcome, through these design decisions."

Roth concludes that "The danger is of inadvertently reading experimental evidence as supporting an overly general conclusion on observations made in special cases." (p.1023)

Despite all this, many of the above objections can be answered (another blog etc etc). Therefore, I still believe that experiments can shed valuable light upon economic theories. And by dealing with the citicisms experimental economics improves in both accuracy and validity. Obviously there are flaws (as with any method of study), but let's not throw the baby out with the over-used metaphor.


Recommended listening:
The Imperial March by John Williams

Key references:

Bardsley, N., R. Cubitt, G. Loomes, P. Moffatt, C. Starmer and R. Sugden (2010)  Experimental Economics: Rethinking the Rules  

Smith, V. L. (1994) "Economics in the Laboratory" Journal of Economic Perspectives, 8, 113-31 

Roth, A. (1988)  "Laboratory Experimentation in Economics: A Methodological Overview", Economic Journal,  974-1031.