This paper reports a first study exploring genomic prediction for adaptation of sorghum [Sorghum bicolor (L.) Moench] to drought-stress (D-ET) and nonstress (W-ET) environment types. The objective was to evaluate the impact of both modeling genotype × environment interaction (G×E) and accounting for heterogeneous variances of marker effects on genomic prediction of parental breeding values for grain yield within and across environment types (ETs). For this aim, different genetic covariance structures and different weights for individual markers were investigated in best linear unbiased prediction (BLUP)-based prediction models. The BLUP models used a kinship matrix combining pedigree and genomic information, termed K-BLUP. The dataset comprised testcross yield performances under D-ET and W-ET as well as pedigree and genomic data. In general, modeling G×E increased predictive ability and reduced empirical bias of genomic predictions for broad adaptation across both ETs vs. models that ignored G×E by fitting a main genetic effect only. Genomic predictions for specific adaptation to D-ET or W-ET were also improved by K-BLUP models that explicitly accommodated G×E and used data from both ETs relative to prediction models that used data from the targeted ET exclusively or models that used all the data but assumed no G×E. Allowing for heterogeneous marker variances through weighted K-BLUP produced clear increments (43–72%) in predictive ability of genomic prediction for grain yield in all adaptation scenarios. We conclude that G×E as well as locus-specific genetic variances should be accommodated in genomic prediction models to improve adaptability of sorghum to variable environmental conditions.