Purpose The aims of this study were to investigate the age-related changes in fatty replacement and cross-sectional area (CSA) of cervical, thoracic, and lumbar paraspinal muscles versus leg muscles in healthy adults and to test for association between muscle fat fraction and lifestyle factors. Methods Fifty-three healthy adults (24-76 yr) were included. Dixon magnetic resonance imaging technique was used to determine CSA and to quantify the fat fraction of paraspinal and leg muscles. Muscle CSA and fat fractions were tested for association with age and muscle strength. The fat fractions were also tested for association with sex, body mass index (BMI), physical activity, and lower back pain. Results Both paraspinal and leg fat fractions correlated directly with age (P < 0.0001). At all ages, fat fraction was higher in paraspinal than leg muscles. The age-related increase in fat fraction was higher in paraspinal muscles than leg muscles (P < 0.0001). The CSA of the muscles did not correlate with age. Knee extension strength correlated with fat fraction (P < 0.05), and the muscle strength of hip muscles, thigh muscles, and anterior calf muscles correlated with CSA (P < 0.05). Sex was associated with lumbar paraspinal fat fraction (P < 0.05) and BMI with thigh fat fraction (P < 0.001). There was no association between fat fraction and physical activity or lower back pain. Conclusion The paraspinal muscles were more susceptible to age-related changes than leg muscles. Further, men had significantly lower fat fractions in lumbar paraspinal muscles, and BMI was positively associated with thigh, but not paraspinal, fat fraction.