Stability of organic photovoltaic devices (OPVs) is a limiting factor for their commercialization and still remains a major challenge whilst power conversion efficiencies are now approaching minimum requirements. The inverted organic solar cell (iOSC) architecture shows promising potential for improving significantly the cell's working lifetime. However, when solution processed ZnO is used as electron extraction layer, an undesirable light-soaking step is commonly required before the device reaches a non-permanent maximum performance. This work investigates the use of Sr and Ba doped ZnO films, ZnSrO and ZnBaO, formed by sol-gel deposition using molecular alkoxide precursor solutions, as electron extraction layers in a model iOSCs with poly [3-hexylthiophene] (P3HT): [6, 6]-phenyl C60 butyl acid methyl ester (PCBM) as the active layer. We show that using these ternary oxides the light-soaking step can be circumvented by preventing a dipole forming between the oxide and the active organic layer as supported by electroabsorption spectroscopy measurements of the device built-in field. It is suggested that Sr or Ba doping results in suppression/reduction of the oxygen adsorption at mobile oxygen vacancy sites on the metal oxide surface. Like in thin film transistor (TFT) applications, where materials like InGaZnO are rapidly becoming an important technology, the use of amorphous, mixed metal oxides allows improving the performance and stability of interfacial charge extraction layers for organic solar cells. Doped ZnO with alkaline earth metals Sr or Ba (ZnSrO, ZnBaO) is used as the electron extraction layer for inverted organic solar cells.