Diabetes mellitus (DM) is an emerging metabolic disease, and the management of diabetic bone disease poses a serious challenge worldwide. Understanding the underlying mechanisms leading to high fracture risk in DM is hence of particular interest and urgently needed to allow for diagnosis and treatment optimization. In a case–control postmortem study, the whole 12th thoracic vertebra and cortical bone from the mid-diaphysis of the femur from male individuals with type 1 diabetes mellitus (T1DM) (n = 6; 61.3 ± 14.6 years), type 2 diabetes mellitus (T2DM) (n = 11; 74.3 ± 7.9 years), and nondiabetic controls (n = 18; 69.3 ± 11.5) were analyzed with clinical and ex situ imaging techniques to explore various bone quality indices. Cortical collagen fibril deformation was measured in a synchrotron setup to assess changes at the nanoscale during tensile testing until failure. In addition, matrix composition was analyzed including determination of cross-linking and non-crosslinking advanced glycation end-products like pentosidine and carboxymethyl-lysine. In T1DM, lower fibril deformation was accompanied by lower mineralization and more mature crystalline apatite. In T2DM, lower fibril deformation concurred with a lower elastic modulus and tendency to higher accumulation of non-crosslinking advanced glycation end-products. The observed lower collagen fibril deformation in diabetic bone may be linked to altered patterns mineral characteristics in T1DM and higher advanced glycation end-product accumulation in T2DM.