Optical coherence tomography (OCT) has been demonstrated as a promising, non-invasive tool in the field of middle ear diagnostics serving for volumetric imaging and phase-sensitive vibrometry. The evaluation of polarization changes induced by birefringent tissue provides an additional contrast, which leads to insights into the sample's microstructure that are indiscernible in the conventional OCT signal. In this study, the morphology of the human tympanic membrane was investigated by polarization-sensitive OCT (PS-OCT) measurements on temporal bone specimens. A differential approach was introduced to determine the depth-resolved optic axis orientation. This revealed the three-dimensional fiber organization of the lamina propria for the first time. Hence, applying PS-OCT for diagnostic purposes of pathological alterations might improve the assessment of the eardrum and, potentially, the middle ear in the future.