Heritable genetic variation in relative brain size can underlie the relationship between brain performance and the relative size of the brain. We used bidirectional artificial selection to study the consequences of genetic variation in relative brain size on brain morphology, cognition and longevity in Nasonia vitripennis parasitoid wasps. Our results show a robust change in relative brain size after 26 generations of selection and 6 generations of relaxation. Total average neuropil volume of the brain was 16% larger in wasps selected for relatively large brains than in wasps selected for relatively small brains, whereas the body length of the large‐brained wasps was smaller. Furthermore, the relative volume of the antennal lobes was larger in wasps with relatively large brains. Relative brain size did not influence olfactory memory retention, whereas wasps that were selected for larger relative brain size had a shorter longevity, which was even further reduced after a learning experience. These effects of genetic variation on neuropil composition and memory retention are different from previously described effects of phenotypic plasticity in absolute brain size. In conclusion, having relatively large brains may be costly for N. vitripennis, whereas no cognitive benefits were recorded.