Women are exposed to a variety of life stressors, particularly violence, during their lifetime which increases the risk of developing various psychiatric and somatic diseases, with the dysregulated secretion of cortisol as one potential biological mechanism. We examined the association between violence and other life stressors and hair cortisol concentration (HCC) in a population of urban women. We included 470 adult women (age = 21–86 years) attending the Cancer Detection Clinic in Iceland. The Life Stressor Checklist-Revised (LSC-R; 30-items) was used to assess exposure. HCC was measured with liquid chromatography coupled with tandem mass spectrometry. We used linear regression models to assess the association between life stressors and log-transformed HCC. The median HCC (pg/mg) in the study population was 4.9 (range 0.6–616.6). HCC was not associated with background covariates, including age (p = 0.868), education level (p = 0.824), marital status (p = 0.545), income (p = 0.363), occupation (p = 0.192), but associated with current smoking (p = 0.013). We noted a 3.3% (95% CI: 0.17–6.6%) associated increase in HCC per endorsed life stressor after adjusting for age and smoking, while non-violent life stressors were not associated with HCC. Per endorsed violence item, we observed a 10.2% (95% CI: 1.4–19.7%) associated increase in HCC after age and smoking adjustment. Women with lifetime exposure to both physical and sexual violence presented with higher HCC than unexposed women (p = 0.010), after age and smoking adjustment. Lifetime exposure to violence was associated with higher levels of HCC in a community sample of women. These findings need confirmation with prospective studies.