Despite the striking increase in the power conversion efficiency (PCE) of lead-based perovskite solar cells (PSCs), their poor operational stability impedes their commercialization. Among the various factors that influence device stability, ion migration has been identified as a key driver of degradation. In this work, the focus is on studying ion migration-induced degradation in inverted architecture PSCs, which employ either a thin polymer layer or a self-assembled monolayer (SAM) for hole extraction. It is demonstrated that the difference in texture imposed by the use of these hole transport layers (HTL) is an important and thus far inconspicuous factor that impacts ion migration, and consequently device stability. By investigating the buried interface in detail, it is revealed that its texture has a strong impact on the vertical compositional stratification in the perovskite active layer. By monitoring bias-induced ion migration in devices with different hole extraction layers, it is demonstrated that the smooth polymer-based HTL results in a higher degree of ion migration than the rough SAM HTL, corresponding to a stronger degradation in the former. These results further indicate that the use of SAMs for hole extraction is a promising strategy to suppress ion migration and improve device efficiency.