We quantify the performance of approximations to stochastic filtering by the Kullback-Leibler divergence to the optimal Bayesian filter. Using a two-state Markov process that drives a Brownian measurement process as prototypical test case, we compare two stochastic filtering approximations: a static low-pass filter as baseline, and machine learning of Volterra expansions using nonlinear Vector Auto-Regression (nVAR). We highlight the crucial role of the chosen performance metric, and present two solutions to the specific challenge of predicting a likelihood bounded between 0 and 1.