The animal and blood-injection-injury (BII) subtypes of specific phobia are both characterized by subjective fear but distinct autonomic reactions to threat. Previous functional neuroimaging studies have related these characteristic responses to shared and non-shared neural underpinnings. However, no comparative structural data are available. This study aims to fill this gap by comparing the two subtypes and also comparing them with a non-phobic control group. Gray and white matter data of 33 snake phobia subjects (SP), 26 dental phobia subjects (DP), and 37 healthy control (HC) subjects were analyzed with voxel-based morphometry. Especially DP differed from HC and SP by showing significantly increased grey matter volumes in widespread areas including the right subgenual anterior cingulate gyrus, left insula, left orbitofrontal and left prefrontal (PFC) cortices. In addition, white matter volume was significantly increased in the left PFC in DP compared with SP. These results are in line with functional changes observed in dental phobia and point toward those brain circuits associated with emotional processing and regulation. Future studies should aim to further delineate functional and structural connectivity alterations in specific phobia.