The control of emotions is of potentially great clinical relevance. Accordingly, there has been increasing interest in understanding the cognitive mechanisms underlying the ability to switch efficiently between the processing of affective and non-affective information. Reports of asymmetrically increased switch costs when switching towards the more salient emotion task indicate specific demands in the flexible control of emotion. The neural mechanisms underlying affective task switching, however, are so far not fully understood. Using functional MRI (N=57), we observed that affective task switching was accompanied by increased activity in domain-general fronto-parietal control systems. BOLD activity in posterior medial frontal and anterolateral prefrontal cortex was directly related to affective switch costs, indicating that these regions play a particular role in individual differences in (affective) task switching ability. Asymmetric switch costs were associated with increased activity in right inferior frontal and dorsal anterior medial prefrontal cortex, two brain regions critical for response inhibition. This suggests that asymmetric switch costs might – to a great extent – reflect higher demands on inhibitory control of the dominant emotion task. These results contribute to a refined understanding of brain systems for the flexible control of emotions, and thereby identify valuable target systems for future clinical research.