Bituminous pavement layers are composite in nature, consisting of binder, filler, and aggregates. Understanding the mechanical behaviour of bituminous layers demands investigation of their different material scales, such as the binder, mastic, and mixture scales. While substantial investigation has been carried out on binder and mixture related to fatigue damage, not much research is carried out on mastic. Of the methods used to characterise fatigue damage, energy dissipation is widely used; however, it is complicated due to the presence of other modes of dissipation, such as viscous dissipation. In this study, an attempt is made to separate viscous dissipation and dissipation due to damage for both binder and mastic using a nonlinear viscoelastic fluid model. For this purpose, six types of materials were tested at temperatures of 20 (Formula presented.) C and 25 (Formula presented.) C, 10 Hz frequency, and five different strain levels spanning the linear and non-linear regimes, for 20,000 cycles. Material parameters for the non-linear White-Metzner model incorporated with the Williamson model were determined for an initial cycle and using these parameters, the viscous component of energy dissipation was determined. Dissipation due to damage was estimated by separating viscous dissipation from total dissipation at every 2000^th cycle and based on the evolution of viscous and damage dissipation across cycles, three trends were observed. Based on the trends observed, the influence of strain amplitude, temperature and modifiers on the fatigue resistance of the materials was ascertained.