Comparably little is known about breast cancer (BC) risks in women from families tested negative for BRCA1/2 mutations despite an indicative family history, as opposed to BRCA1/2 mutation carriers. We determined the age-dependent risks of first and contralateral breast cancer (FBC, CBC) both in noncarriers and carriers of BRCA1/2 mutations, who participated in an intensified breast imaging surveillance program. The study was conducted between January 1, 2005, and September 30, 2017, at 12 university centers of the German Consortium for Hereditary Breast and Ovarian Cancer. Two cohorts were prospectively followed up for incident FBC (n = 4,380; 16,398 person-years [PY], median baseline age: 39 years) and CBC (n = 2,993; 10,090 PY, median baseline age: 42 years). Cumulative FBC risk at age 60 was 61.8% (95% CI 52.8–70.9%) for BRCA1 mutation carriers, 43.2% (95% CI 32.1–56.3%) for BRCA2 mutation carriers and 15.7% (95% CI 11.9–20.4%) for noncarriers. FBC risks were significantly higher than in the general population, with incidence rate ratios of 23.9 (95% CI 18.9–29.8) for BRCA1 mutation carriers, 13.5 (95% CI 9.2–19.1) for BRCA2 mutation carriers and 4.9 (95% CI 3.8–6.3) for BRCA1/2 noncarriers. Cumulative CBC risk 10 years after FBC was 25.1% (95% CI 19.6–31.9%) for BRCA1 mutation carriers, 6.6% (95% CI 3.4–12.5%) for BRCA2 mutation carriers and 3.6% (95% CI 2.2–5.7%) for noncarriers. CBC risk in noncarriers was similar to women with unilateral BC from the general population. Further studies are needed to confirm whether less intensified surveillance is justified in women from BRCA1/2 negative families with elevated risk.