Vascular graft/endograft infection (VGEI) is a serious complication after aortic surgery. This study investigates VGEI and patient characteristics, PET/CT quantification before surgical or conservative management of VGEI and post-intervention outcomes in order to identify patients who might benefit from such a procedure. PET standard uptake values (SUV) were quantitatively assessed and compared to a non-VGEI cohort. The primary endpoints were in-hospital mortality and aortic reintervention-free survival at six months. Ninety-three patients (75% male, 65 ± 10 years, 82% operated) were included. The initial operation was mainly for aneurysm (67.7%: 31% EVAR, 12% TEVAR, 57% open aortic repair). Thirty-two patients presented with fistulae. PET SUV_TLR (target-to-liver ratio) showed 94% sensitivity and 89% specificity. Replacement included silver-coated Dacron (21.3%), pericardium (61.3%) and femoral vein (17.3%), yet the material did not influence the overall survival (p = 0.745). In-hospital mortality did not differ between operative and conservative treatment (19.7% vs. 17.6%, p = 0.84). At six months, 50% of the operated cohort survived without aortic reintervention. Short- and midterm morbidity and mortality remained high after aortic graft removal. Neither preoperative characteristics nor the material used for reconstruction influenced the overall survival, and, with limitations, both the in-hospital and midterm survival were similar between the surgically and conservatively managed patients.