Encouraging adaptation to climate change is fundamentally about encouraging changes in human behaviour. To promote these changes, governments, non-profits and multilateral institutions have invested in a range of adaptation projects. Yet there is little empirical evidence about which project components are effective in changing human behaviour1,2. This lack of evidence is concerning, given that the failure of adaptation initiatives has been described as the global risk with the highest likelihood of occurring and with the largest negative impacts3. Here we report on a scholar–practitioner collaboration in which a simple one-day workshop delivering two ubiquitous components of adaptation projects4—capacity building and the dissemination of climate science—was randomly assigned among the management councils of over 200 community water systems in an arid region of Central America. The workshop was based on more than three years of scientific research and local collaborations, and it aimed to convey downscaled climate modelling and locally informed, expert-recommended adaptation practices. Two years later, we detect no differences in pricing and non-pricing management practices of participant versus non-participant councils. These results suggest weaknesses in the common practice of using simple workshops for delivering capacity building and climate science.