Based on a time-use model with a sound theoretical basis and carefully collected data for Austria, the value of leisure (VoL) for different population segments has been estimated. Through the combination of these results with mode-specific values of travel time savings from a related study based on the same data, the first mode-specific values of time assigned to travel (VTAT) were calculated. Data was collected using a Mobility-Activity-Expenditure Diary, a novel survey format which gathers all activities, expenditures, and travel decisions from the same individuals for 1 week in a diary-based format. The average VoL is 8.17 €/h, which is below the mean wage of 12.14 €/h, indicating that the value of work is, on average, negative. Regarding the reliability of the VoL, we show its sensitivity to the variance of working time in a sample, something that has been ignored in previous studies and could be used to avoid inadequate segmentation. We controlled this effect in the analysis of the heterogeneity of the VoL across the population by estimating the parameters from the total (unsegmented) dataset with single interaction terms. We find that the VTAT is strictly negative for walking, predominantly negative for cycling and car, and predominantly positive for public transport with 0.27 €/h on average. The positive VTAT for public transport is a strong indication for the importance of travel conditions, in turn suggesting that improvements in travel conditions of public transport might be as important as investing in shorter travel times.