Recent research widely deployed Neural Networks (NNs) in various scenarios, such as IoT systems, wearable devices, or smart sensors. However, the complex application scenarios cause the rapid extension of network model size and the requirement for higher-performance hardware platforms. Related works apply Heterogeneous Streaming Dataflow (HSD) and Processing Element Matrix (PEM) architectures as the most popular schemes for FPGA-based implementation of NN accelerator: 1) HSD architecture implements a complete network for given trained models on FPGA with simplified control but more hardware consumption; 2) PEM architecture implements reusable neuron structures controlled by runtime environments/drivers providing the generic acceleration supports for different network models. Our work explores a new hybrid architecture based on HSD and PEM to implement a reusable partial network structure on FPGA and achieve generic acceleration supports for different network models with simplffied runtime control. This architecture supports scalable, mixable, quantized precision, and selectable activation functions, including ReLU, Sigmoid, Tanh, Sign, and Multi-Thresholds. Data stream transmission can reset the accelerator configuration in runtime without hardware implementation changes for different networks. Our design fully supports the generic inference acceleration for different Multi-Layer Perceptron (MLP) models.