Pyrrhotite, Fe$_7$S$_8$, provides an example of exceptionally strong magnetoelastic coupling through pinning of ferromagnetic domains by ferroelastic twins. Using direct imaging of both magnetic and ferroelastic domains by magnetic force microscopy (MFM), the mechanism by which this coupling controls local magnetic switching behaviour of regions on the pyrrhotite surface is revealed, and leads to quantitative fitting of field dependent MFM phase shifts with bulk magnetometry data. It is shown that characteristic inflection points in the magnetometry data along certain direction, in particular $[\overline 120]^*_h$ of the hexagonal parent structure, are in fact caused by ferroelastic pinning of the magnetic moments.