We present a scalable architecture based on a trained filter bank for input pre-processing and a recurrent neural network (RNN) for the detection of atrial fibrillation in electrocardiogram (ECG) signals, with the focus on enabling a very efficient hardware implementation as application-specific integrated circuit (ASIC). Our already very efficient base architecture is further improved by replacing the RNN with a delta-encoded gated recurrent unit (GRU) and adding a confidence measure (CM) for terminating the computation as early as possible. With these optimizations, we demonstrate a reduction of the processing load of 58 % on an internal dataset while still achieving near state-of-the-art classification results on the Physionet ECG dataset with only 1202 parameters.