Soil degradation is a major threat to Southern Africa's agricultural production. Crops show generally weak responses to mineral fertilizers on degraded soils. A three-year study was conducted between 2009 and 2012 on smallholder farms in eastern Zimbabwe to explore entry points for rehabilitating degraded croplands using principles of integrated soil fertility management (ISFM) supported through farmers' local knowledge of soils. Participatory research approaches were first used to investigate farmers' understanding of soil degradation and the commonly used local diagnostic indicators. Farmers' determinants of degraded soils centered on crop performance, indicator weed species and soil physical attributes, and matched laboratory parameters. Overall, physical and chemical properties of the degraded soils were significantly lower than reported values for productive sandy soils in Zimbabwe. Evaluated on ten degraded field sites of corresponding catenary positions and similar slope, the main ISFM options involved nitrogen-fixing herbaceous legumes planted in the first year, with subsequent addition of cattle manure in the second year. In the third year, the influence of the ISFM options on maize productivity and changes in soil biological activity were then evaluated. Phosphorus was applied every year under each sequence. The controls were natural fallow and continuous maize. The treatments were randomly assigned to plots at each of the experimental sites and replicated across farms. Above-ground biomass carbon (C) and nitrogen (N) accumulation was 3038 kg ha- 1 and 203 kg ha- 1, respectively, under 1-year indigenous legume fallow (indifallow) against 518 kg C ha- 1 and 14 kg N ha- 1 under 1-year natural fallow. Two-year indifallow produced approximately three times the biomass N attained under the 2-year natural fallow. When all the treatments were planted to a maize test crop in the third year, herbaceous legume-based sequences showed the highest response to mineral fertilizer N compared with natural fallow-based sequences and continuous fertilized maize. A regression of maize yields against mineral N fertilizer showed a maximum yield of 2.5 t ha- 1 under the herbaceous legume-based sequences against 1 t ha- 1 under continuous fertilized maize and natural fallow-based options following addition of 120 kg ha- 1 of mineral N fertilizer. ‘Green-start’, a Crotalaria juncea L. (sunnhemp)-based sequence, and ‘Indifallow-start 1’, an indigenous legume-based sequence, gave the highest microbial biomass C (MBC) of 243 mg kg- 1 soil compared with 187 mg kg- 1 soil under continuous maize. Microbial biomass N showed a similar trend. Under ‘Green-start’ and ‘Indifallow-start 1’, MBC to organic C ratio averaged 7; about one and half times more than under natural fallow-based sequences and continuous fertilized maize. Consistent with microbial biomass, soil carbon dioxide (CO2) emission under ‘Green-start’ and ‘Indifallow-start 1’ was 22% higher than under natural fallow-based sequences. Continuous maize treatments gave higher metabolic quotients (qCO2) than legume-based sequences, indicating a lower microbial efficiency under the former. We concluded that short-term restoration of productivity of degraded sandy soils should focus on high quality organic resource application and P fertilization to stimulate microbial activity and induce responses to mineral fertilizers. When coupled to P fertilization, herbaceous legume-based ISFM sequences provide a potential entry point for reversing soil degradation and offer opportunities for increasing crop productivity in dominant smallholder farming systems of Zimbabwe and other parts of Southern Africa.