How spiking neuronal networks encode memories in their different time and spatial scales constitute a fundamental topic in neuroscience and neuro-inspired engineering. Much attention has been paid to large networks and long-term memory, for example in models of associative memory. Smaller circuit motifs may play an important complementary role on shorter time scales, where broader network effects may be of less relevance. Yet, compact computational models of spiking neural networks that exhibit short-term volatile memory and actively hold information until their energy source is switched off, seem not fully understood. Here we propose that small spiking neural circuit motifs may act as volatile memory components. A minimal motif consists of only two interconnected neurons - one self-connected excitatory neuron and one inhibitory neuron - and realizes a single-bit volatile memory. An excitatory, delayed self-connection promotes a bistable circuit in which a self-sustained periodic orbit generating spike trains co-exists with the quiescent state of no neuron spiking. Transient external inputs may straightforwardly induce switching between those states. Moreover, the inhibitory neuron may act as an autonomous turn-off switch. It integrates incoming excitatory pulses until a threshold is reached after which the inhibitory neuron emits a spike that then inhibits further spikes in the excitatory neuron, terminating the memory. Our results show how external bits of information (excitatory signal), can be actively held in memory for a pre-defined amount of time. We show that such memory operations are robust against parameter variations and exemplify how sequences of multidimensional input signals may control the dynamics of a many-bits memory circuit in a desired way.