Walking as one type of physical activity generates benefits for personal health and contributes to sustainability in its environmental, economic, and social dimensions. Based on a cross-sectional survey for a representative sample of German cities with at least 100,000 residents, this study investigates determinants of residents’ walking behavior. Two contexts of promenading in the city for leisure walking and the trip to the supermarket for utilitarian walking are distinguished in order to investigate differences in respondents’ perceived importance of built-environment characteristics and to bridge the gap between the individual level of residents’ behavior and the streetscape level. In addition, the analyses distinguish between frequent and less frequent walkers in order to understand differences in characteristics, behavior, preferences, and perceptions between these two groups. The results of this study with a sample of n = 4,637 respondents show that the relevance of the built environment is higher for leisure walking than for utilitarian walking and higher for frequent walkers compared to less frequent walkers. For leisure walking, “protection”-variables are ranked high (e.g. safety, security), followed by “comfort” (e.g. space, surface quality) and by “delight” (e.g. attractive buildings, greenery). Distance is key for utilitarian walking. Significant differences between frequent and less frequent walkers are identified mainly for the “Delight”-variables, this is attractive buildings and greenery for utilitarian walking, and space availability and street furniture for leisure walking. These differences should be considered in future research and street design practice which might preferably focus on the most demanding person group of frequent walkers.