The implementation of modern complex machine learning algorithms on a conventional von Neumann architecture faces severe challenges to maintain efficiency in terms of energy consumption, memory requirement, and latency. Data-centric architectures result in a more appropriate alternative to exploit the inherent parallelism of multidimensional sensory signals. A viable implementation alternative is based on memristor crossbars. Matrix-vector multiplication, which is one resource-hungry operation in conventional architectures, can be performed by memristor crossbar arrays by using very limited resources. In this work, we explore the implementation of a multiclass support vector machine (SVM) on a differential memristor crossbar array. The Voltage Threshold Adaptive Memristor (VTEAM) model has been employed for the simulation of memristor behavior. The feature vector for SVM has been produced by a compressed sensing based vision sensor architecture. The weights of the ex situ trained SVM, are mapped to the device physical parameters to achieve the equivalent conductance. As a case study, the proposed scheme has been tested for face recognition against a widely accepted face dataset.