The so-called Transport of Intensity Equation (TIE) phase retrieval technique is widely applied in light, X-ray and electron optics to reconstruct, e.g., refractive indices, electric and magnetic fields in solids. Here, we present a largely improved TIE reconstruction algorithm, which properly considers intensity variations as well as unknown boundary conditions in a finite difference implementation of the Transport of Intensity partial differential equation. That largely removes reconstruction artifacts encountered in state-of-the-art Poisson solvers of the TIE, and hence significantly increases the applicability of the technique.