Randomized interventions and “real” treatment effects

A cautionary tale and an example

Erwin Bulte*, Salvatore Di Falco, Robert Lensink

*Corresponding author for this work

Research output: Contribution to journalComment/Letter to the editorAcademic

Abstract

The experimental approach has revolutionized development economics. Nonetheless, randomization cannot do everything. We discuss challenges to RCTs, paying special attention to internal validity. Randomized interventions in social sciences are not double-blind and do not, in general, hold all relevant covariates constant. Treated and untreated subjects adjust their behavior in response to treatment status. Disentangling the treatment effect into its behavioral component and the direct effect of the intervention is difficult, and implies a return to the toolkit of observational studies. This is illustrated using improved seed distribution in African farming. While standard RCTs found large treatment effects, double-blind RCTs revealed that a large share of this impact is due to farmers allocating extra effort and their best plots to the cultivation of new seeds.

Original languageEnglish
Article number104790
JournalWorld Development
Volume127
DOIs
Publication statusPublished - 1 Mar 2020

Fingerprint

seed
development economics
farmer
social science
economics
effect
Randomized controlled trial
Treatment effects
distribution
Randomization
Observational study
Farming
Toolkit
Internal validity
Direct effect
Social sciences
Covariates
Farmers
Development economics
Africa

Cite this

@article{574579cb3fbe4e98a3b875be3c4e5dc3,
title = "Randomized interventions and “real” treatment effects: A cautionary tale and an example",
abstract = "The experimental approach has revolutionized development economics. Nonetheless, randomization cannot do everything. We discuss challenges to RCTs, paying special attention to internal validity. Randomized interventions in social sciences are not double-blind and do not, in general, hold all relevant covariates constant. Treated and untreated subjects adjust their behavior in response to treatment status. Disentangling the treatment effect into its behavioral component and the direct effect of the intervention is difficult, and implies a return to the toolkit of observational studies. This is illustrated using improved seed distribution in African farming. While standard RCTs found large treatment effects, double-blind RCTs revealed that a large share of this impact is due to farmers allocating extra effort and their best plots to the cultivation of new seeds.",
author = "Erwin Bulte and {Di Falco}, Salvatore and Robert Lensink",
year = "2020",
month = "3",
day = "1",
doi = "10.1016/j.worlddev.2019.104790",
language = "English",
volume = "127",
journal = "World Development",
issn = "0305-750X",
publisher = "Elsevier",

}

Randomized interventions and “real” treatment effects : A cautionary tale and an example. / Bulte, Erwin; Di Falco, Salvatore; Lensink, Robert.

In: World Development, Vol. 127, 104790, 01.03.2020.

Research output: Contribution to journalComment/Letter to the editorAcademic

TY - JOUR

T1 - Randomized interventions and “real” treatment effects

T2 - A cautionary tale and an example

AU - Bulte, Erwin

AU - Di Falco, Salvatore

AU - Lensink, Robert

PY - 2020/3/1

Y1 - 2020/3/1

N2 - The experimental approach has revolutionized development economics. Nonetheless, randomization cannot do everything. We discuss challenges to RCTs, paying special attention to internal validity. Randomized interventions in social sciences are not double-blind and do not, in general, hold all relevant covariates constant. Treated and untreated subjects adjust their behavior in response to treatment status. Disentangling the treatment effect into its behavioral component and the direct effect of the intervention is difficult, and implies a return to the toolkit of observational studies. This is illustrated using improved seed distribution in African farming. While standard RCTs found large treatment effects, double-blind RCTs revealed that a large share of this impact is due to farmers allocating extra effort and their best plots to the cultivation of new seeds.

AB - The experimental approach has revolutionized development economics. Nonetheless, randomization cannot do everything. We discuss challenges to RCTs, paying special attention to internal validity. Randomized interventions in social sciences are not double-blind and do not, in general, hold all relevant covariates constant. Treated and untreated subjects adjust their behavior in response to treatment status. Disentangling the treatment effect into its behavioral component and the direct effect of the intervention is difficult, and implies a return to the toolkit of observational studies. This is illustrated using improved seed distribution in African farming. While standard RCTs found large treatment effects, double-blind RCTs revealed that a large share of this impact is due to farmers allocating extra effort and their best plots to the cultivation of new seeds.

U2 - 10.1016/j.worlddev.2019.104790

DO - 10.1016/j.worlddev.2019.104790

M3 - Comment/Letter to the editor

VL - 127

JO - World Development

JF - World Development

SN - 0305-750X

M1 - 104790

ER -