Significance: Endoscopic optical coherence tomography (OCT) is of growing interest for in vivo diagnostics of the tympanic membrane (TM) and the middle ear but generally lacks a tissue-specific contrast. Aim: To assess the collagen fiber layer within the in vivo TM, an endoscopic imaging method utilizing the polarization changes induced by the birefringent connective tissue was developed. Approach: An endoscopic swept-source OCT setup was redesigned and extended by a polarization-diverse balanced detection unit. Polarization-sensitive OCT (PS-OCT) data were visualized by a differential Stokes-based processing and the derived local retardation. The left and right ears of a healthy volunteer were examined. Results: Distinct retardation signals in the annulus region of the TM and near the umbo revealed the layered structure of the TM. Due to the TM's conical shape and orientation in the ear canal, high incident angles onto the TM's surface, and low thicknesses compared to the axial resolution limit of the system, other regions of the TM were more difficult to evaluate. Conclusions: The use of endoscopic PS-OCT is feasible to differentiate birefringent and nonbirefringent tissue of the human TM in vivo. Further investigations on healthy as well as pathologically altered TMs are required to validate the diagnostic potential of this technique.