Locally active memristors (LAMs) exhibit small-signal amplification capability, making them suitable for use in artificial neuron circuits. Spiking oscillations and chaotic dynamics are two representative neuromorphic behaviors that have shown promise in spiking neural networks and combinatorial optimization applications. Spiking oscillations are identified using a newly proposed criterion based on the signal's rate of change and energy consumption characteristics, while chaotic dynamics are verified through Lyapunov exponent analysis. To investigate their underlying mechanisms, simple second-order and third-order memristive neuron circuits are employed to generate periodic spiking and chaotic neuromorphic behaviors, respectively. Based on nonlinear circuit and dynamics theory as well as numerical analysis methods, the impacts of model expressions and parameters on spiking oscillation and chaotic behavior are quantitatively investigated. The analysis results indicate that the emergence of these two neuromorphic behaviors mainly depends on the expression of memristance/memductance functions in the LAMs polynomial model and the characteristics of the instantaneous resistance and the differential resistance of the LAMs at the operating point. Hardware implementations of both circuits further validate the theoretical and simulation results. This insight provides valuable guidance for designing and optimizing neuron models and neuromorphic computing devices, advancing the realization of circuit-oriented neuromorphic computing systems.