Thermoelectric materials exhibit correlated transport of charge and heat. The Johnson-Nyquist noise formula 4k _B T R for the spectral density of voltage fluctuations accounts for fluctuations associated solely with Ohmic dissipation. Applying the fluctuation-dissipation theorem, we generalize the Johnson-Nyquist formula for thermoelectrics, finding an enhanced voltage fluctuation spectral density 4k _B T R(1 + Z _D T) at frequencies below a thermal cut-off frequency f _T, where Z _D T is the dimensionless thermoelectric device figure of merit. The origin of the enhancement in voltage noise is thermoelectric coupling of temperature fluctuations. We use a wideband , integrated thermoelectric micro-device to experimentally confirm our findings. Measuring the Z _D T enhanced voltage noise, we experimentally resolve temperature fluctuations with a root mean square amplitude of at a mean temperature of 295 K. We find that thermoelectric devices can be used for thermometry with sufficient resolution to measure the fundamental temperature fluctuations described by the fluctuation-dissipation theorem.