A NbOx and Al2O3 bilayer metal-insulator-metal structure features capacitive state change together with gradual, non-filamentary resistive switching. The reported device offers several benefits, including no need for electroforming and intrinsic current compliance. These features are highly attractive for neuromorphic computing applications, where biological plasticity can be emulated by not only memristance but also memcapacitance of the device. To investigate the capability of a Ti/Al2O3/NbOx/Ti stack to produce the behavior necessary for mimicking the functionality of neurons or synapses, its dynamic response is studied in detail by means of pulsed I-V and C-V measurements. Additionally, the correlation between the capacitive and the memristive state change in the stack is discussed.