Resorbable biomaterials based on artificial extracellular matrices (aECM) represent promising scaffolds for the treatment of large bone defects. Here, we investigated various glycosaminoglycan (GAG) derivatives of varying sulfation degree with respect to their influence on in vivo bone healing. The materials used in this study consisted of GAG-coated degradable polycaprolactone-co-lactide (PCL). Critical size femur defects in rats were filled with autologous bone serving as positive control or the respective coated or uncoated PCL scaffolds. After 2 and 12 weeks, progress in the healing process was investigated by analyzing the new bone matrix formation, the collagen content and hydroxyapatite formation by using micro-computed tomography (μCT), biomechanical testing, nuclear magnetic resonance spectroscopy (NMR) and histology. The sulfated GAG coating contributed substantially to bone regeneration, increased collagen synthesis and initiated mineralization of the organic matrix. Most substantial collagen production was detected in scaffolds coated with chondroitin sulfate. Scaffolds coated with hypersulfated hyaluronan induced formation of new bone volume comparable to what was observed in the positive control. GAG differing in the sugar backbone and degree of sulfation modulate the healing process at different times, eventually leading to improved bone healing.