Previous studies have indicated that both low physical activity and low physical fitness are associated with a higher level of stress but the influence of age and health status on the associations is unknown. This was examined in a cross-sectional study based on data from the Danish Capital Region Health Survey 2017. Among all adults ≥ 16 years residing in the largest of five regions in Denmark 1. January 2017 a random sample of 104,950 was invited to participate. Hereof, 55.185 responded (52,6%). Physical activity during leisure time, fitness, self-rated health and stress (Cohens Perceived Stress Scale) was self-reported by questionnaire. Logistic regression weighted for size of municipality and non-response was used. Age modified the associations. In all age-groups odds ratio (OR) for a high level of stress was increasingly higher the lower the level of physical activity. The association was strongest among the 16-24-year-olds and persisted after adjustment for self-rated health, that otherwise attenuated the associations to an increasing extent the older the age-group. Similar models investigating the modifying effect of age on the association between self-rated fitness and stress showed the same patterns and tendencies. This study showed that physical activity and self-rated fitness were both associated with stress. The OR for a high level of stress was increasingly higher the lower the level of physical activity or self-rated fitness. This was found in all age-groups, but most pronounced among the 16-24-year-olds. Furthermore, findings suggest that health condition explains the associations to an increasing extent with increasing age.