Lies, Damned Lies, and Crafty Questionnaire Design

It is well-established in the literature that particular design features of questionnaires affect the distribution and association of collected data. We present a survey approach called Crafty Questionnaire Design (CQED), that allows predictability and replicability of outcomes, expected of the natural sciences, to be achieved in the social sciences. Two independent proof-of-principle experiments
studying interpersonal and institutional trust of Polish and Mexican students (n =1402), show that using different versions of a questionnaire offers predictably different outcomes. CQED promises a large gain in efficiency of research in terms of sample size required and number of replications needed. This knowledge can safeguard the social scientific researcher against unpleasant surprises and inconvenient results. Knowledge about the principles of CQED could also be a tool for editors as well as reviewers of social scientific journals to scrutinize the methodological soundness and improve the relevance of publications.