Despite increasing attention for the recently found ferro- and antiferroelectric properties, the polymorphism in hafnia- and zirconia-based thin films is still not sufficiently understood. In the present work, we show that it is important to have a good quality X-ray absorption spectrum to go beyond an analysis of the only the first coordination shell. Equally important is to analyze both EXAFS and XANES spectra in combination with theoretical modelling to distinguish the relevant phases even in bulk materials and to separate structural from chemical effects. As a first step toward the analysis of thin films, we start with the analysis of bulk references. After that, we successfully demonstrate an approach that allows us to extract high-quality spectra also for 20 nm thin films. Our analysis extends to the second coordination shell and includes effects created by chemical substitution of Hf with Zr to unambiguously discriminate the different polymorphs. The trends derived from X-ray absorption spectroscopy agree well with X-ray diffraction measurements. In this work we clearly identify a gradual transformation from monoclinic to tetragonal phase as the Zr content of the films increases. We separated structural effects from effects created by chemical disorder when ration of Hf:Zr is varied and found differences for the incorporation of the substitute atoms between powders and thin films, which we attribute to the different fabrication routes. This work opens the door for further in-depth structural studies to shine light into the chemistry and physics of these novel ferroelectric thin films that show high application relevance.