Nelder-Mead algorithm (NMA) is the best-known algorithm for multidimensional optimization without involving derivative computations. Due to the simplicity in implementation and the fast convergent property of NMA, it is widely used in the fields of statistics, engineering, physics and medical sciences. In practice, when objective function is complicated, the optimization procedure requires a lot of computation efforts, leading to a time-consuming process. This work introduces a NMA solver engine fully implemented on FPGA hardware and performs design space exploration to provide various solutions suitable for FPGA device