Structural strain severely impacts material properties such as the linear and non-linear optical response. Moreover, strain plays a key role, e.g., in the physics of ferroelectrics and in particular of their domain walls. $\mu$-Raman spectroscopy is a well-suited technique for the investigation of such strain effects, as it allows to measure the lattice dynamics locally. However, quantifying and reconstructing strain fields from Raman maps requires knowledge on the strain dependence of phonon frequencies. In this work, we have analyzed both theoretically and experimentally the phonon frequencies in the widely used ferroelectrics lithium niobate and lithium tantalate as a function of uniaxial strain via density functional theory and $\mu$-Raman spectroscopy. Overall, we find a good agreement between our $ab$ $initio$ models and the experimental data performed with a stress cell. The majority of phonons show an increase in frequency under compressive strain, while the opposite is observed for tensile strains. Moreover, for E-type phonons, we observe the lifting of degeneracy already at moderate strain fields (i.e. at $\pm0.2~\%$) along the x and y directions. This work hence allows for the systematic analysis of 3D strains in modern-type bulk and thin-film devices assembled from lithium niobate and tantalate.