The last five years have witnessed remarkable progress in the field of lead halide perovskite materials and devices. Examining the existing body of literature reveals staggering inconsistencies in the reported results among different research groups with a particularly wide spread in the photovoltaic performance and stability of devices. In this work we demonstrate that fractional, quite possibly unintentional, deviations in the precursor solution stoichiometry can cause significant changes in the properties of the perovskite layer as well as in the performance and stability of perovskite photovoltaic devices. We show that while the absorbance and morphology of the layers remain largely unaffected, the surface composition and energetics, crystallinity, emission efficiency, energetic disorder and storage stability are all very sensitive to the precise stoichiometry of the precursor solution. Our results elucidate the origin of the irreproducibility and inconsistencies of reported results among different groups as well as the wide spread in device performance even within individual studies. Finally, we propose a simple experimental method to identify the exact stoichiometry of the perovskite layer that researchers can employ to confirm their experiments are performed consistently without unintentional variations in precursor stoichiometry.