The Heterogeneous Autoregressive (HAR) model is commonly used in modeling the dynamics of realized volatility. In this paper, we propose a flexible HAR(1, . . . , p) specification, employing the adaptive LASSO and its statistical inference theory to see whether the lag structure (1, 5, 22) implied from an economic point of view can be recovered by statistical methods. The model differs from Audrino and Knaus (2016) [Audrino, F. and S. D. Knaus. 2016. "Lassoing the HAR model: A model selection perspective on realized volatility dynamics." Econometrics Review 35: 1485-1521]. where the authors apply LASSO on the AR(p) model, which does not necessarily lead to a HAR model. Adaptive LASSO estimation and the subsequent hypothesis testing results fail to show strong evidence that such a fixed lag structure can be recovered by a flexible model. We also apply the group LASSO and related tests to check the validity of the classic HAR, which is rejected in most cases. The results justify our intention to use a flexible lag structure while still keeping the HAR frame. In terms of the out-of-sample forecasting, the proposed flexible specification works comparably to the benchmark HAR(1, 5, 22). Moreover, the time-varying model combinations show that when the market environment is not stable, the fixed lag structure (1, 5, 22) is not particularly accurate and effective.