For crops that are grown for their fruits or seeds, elevated temperatures that occur during flowering and seed or fruit set have a stronger effect on yield than high temperatures during the vegetative stage. Even short-term exposure to heat can have a large impact on yield. In this study, we used Arabidopsis thaliana to study the effect of short-term heat exposure on flower and seed development. The impact of a single hot day (35°C) was determined in more than 250 natural accessions by measuring the lengths of the siliques along the main inflorescence. Two sensitive developmental stages were identified, one before anthesis, during male and female meiosis, and one after anthesis, during fertilization and early embryo development. In addition, we observed a correlation between flowering time and heat tolerance. Genome-wide association mapping revealed four quantitative trait loci (QTLs) strongly associated with the heat response. These QTLs were developmental stage specific, as different QTLs were detected before and after anthesis. For a number of QTLs, T-DNA insertion knockout lines could validate assigned candidate genes. Our findings show that the regulation of complex traits can be highly dependent on the developmental timing.