The building energy (BE) management plays an essential role in urban sustainability and smart cities. Recently, the novel data science and data-driven technologies have shown significant progress in analyzing the energy consumption and energy demand datasets for a smarter energy management. The machine learning (ML) and deep learning (DL) methods and applications, in particular, have been promising for the advancement of accurate and high-performance energy models. The present study provides a comprehensive review of ML- and DL-based techniques applied for handling BE systems, and it further evaluates the performance of these techniques. Through a systematic review and a comprehensive taxonomy, the advances of ML and DL-based techniques are carefully investigated, and the promising models are introduced. According to the results obtained for energy demand forecasting, the hybrid and ensemble methods are located in the high-robustness range, SVM-based methods are located in good robustness limitation, ANN-based methods are located in medium-robustness limitation, and linear regression models are located in low-robustness limitations. On the other hand, for energy consumption forecasting, DL-based, hybrid, and ensemble-based models provided the highest robustness score. ANN, SVM, and single ML models provided good and medium robustness, and LR-based models provided a lower robustness score. In addition, for energy load forecasting, LR-based models provided the lower robustness score. The hybrid and ensemble-based models provided a higher robustness score. The DL-based and SVM-based techniques provided a good robustness score, and ANN-based techniques provided a medium robustness score.