The random thermal force acting on Brownian particles is often approximated in Langevin models by a "white-noise" process. However, fluid entrainment results in a frequency dependence of this thermal force giving it a "color." While theoretically well understood, direct experimental evidence for this colored nature of the noise term and how it is influenced by a nearby wall is lacking. Here, we directly measured the color of the thermal noise intensity by tracking a particle strongly confined in an ultrastable optical trap. All our measurements are in quantitative agreement with the theoretical predictions. Since Brownian motion is important for microscopic, in particular, biological systems, the colored nature of the noise and its distance dependence to nearby objects need to be accounted for and may even be utilized for advanced sensor applications.