In complex operational and environmental conditions, structural vibration signals are typically characterized by inherent non-stationarity and nonlinearity. Traditional convolutional neural networks (CNNs) with single-feature extraction struggle to comprehensively capture the multiscale spatiotemporal dependencies inherent in complex damage patterns. To address these limitations, this study proposes a multidomain feature fusion-based gated recurrent unit-transformer (DF-GRU-Transformer) framework for efficient structural damage identification. An experimental study on a scaled steel truss bridge was conducted to validate the effectiveness of the proposed framework under various damage conditions. Furthermore, a case study on the Old ADA Bridge in Japan was carried out to demonstrate the applicability of the method to real bridge scenarios. Comparative analyses among three classic networks and five damage conditions were performed to demonstrate the performance of the proposed framework. Experimental results indicated that the DF-GRU-Transformer model achieves superior accuracy, precision, F1-score, and lower average loss compared with other benchmark models across five representative conditions. The multidomain feature fusion effectively characterized the multidimensional information and reduced interclass confusion. The t-distributed stochastic neighbor embedding (t-SNE) visualization further confirmed the advanced feature extraction capability and interclass separability in high-dimensional feature space. The on-site validation on the full-scale bridge demonstrated that the proposed approach can effectively decouple environmental noise from structural responses.