The ferroelectric properties of hafnium oxide films are strongly influenced by the crystallization process due to the interaction of thermodynamics, kinetics, and mechanical stress. In this work, the influence of annealing temperature on the crystallographic properties and microstructure of Si-doped hafnium oxide thin films as well as their ferroelectric properties are investigated by X-ray diffraction, transmission Kikuchi diffraction, and electrical characterization. The findings reveal the emergence of a [100] and [110] out-of-plane texture for metal-ferroelectric-metal (MFM) and metal-ferroelectric-insulator-semiconductor (MFIS) capacitor structures with increasing annealing temperature, respectively. In combination with observed stress relaxation at higher temperatures and the evolution of the wake-up behavior, insights into the crystallization process and the influence of the interplay of microstructure and stress on the ferroelectric properties of hafnium oxide thin films are given.