Background: We aimed to assess the association of waist circumference (WC) and body mass index (BMI) with health-related quality of life (HRQL) among colorectal cancer (CRC) survivors. Methods: CRC survivors diagnosed between 2000 and 2009 completed questionnaires in August 2013 (with self-reported weight, height, and self-assessed WC) and January 2014 (with HRQL using the EORTC-QLQ-C30). Clinical characteristics were retrieved from the Netherlands Cancer Registry. In multivariable linear regression analyses associations of BMI only, WC only and both BMI and WC with HRQL outcomes were assessed. Results: 1,111 CRC survivors were included of whom 34% had a normal weight (18.5 ≤ BMI < 25 kg/m2), 49% had overweight (25 ≤ BMI < 30 kg/m2), 17% had obesity (BMI ≥ 30 kg/m2), and 44% had an increased WC (i.e., >102 and >88 cm for men and women, respectively). Both BMI and WC were separately associated with worse global health status, functioning, and more symptoms of fatigue. Increased WC was associated with lower physical, role and emotional functioning, regardless of BMI, with average differences ranging between 3 and 5 points. Conclusion: Future research on HRQL among CRC survivors should consider both BMI and WC. Furthermore, weight reduction trials should not only focus on general weight loss but also on the loss of abdominal fat.