Background: Substantial evidence links activity domains with health and well-being; however, research has typically examined time-use behaviors independently, rather than considering daily activity as a 24-hour time-use composition. This study used compositional data analysis to estimate the difference in physical and mental well-being associated with reallocating time between behaviors. Methods: Participants (n = 430; 74% female; 41  y) wore an accelerometer for 7 days and reported their body mass index; health-related quality of life (QoL); and symptoms of depression, anxiety, and stress. Regression models determined whether time-use composition, comprising sleep, sedentary behavior, light physical activity (LPA), and moderate to vigorous physical activity (MVPA), was associated with well-being. Compositional isotemporal substitution models estimated the difference in well-being associated with reallocating time between behaviors. Results: Time-use composition was associated with body mass index and physical health-related QoL. Reallocating time to MVPA from sleep, sedentary behavior, and LPA showed favorable associations with body mass index and physical health-related QoL, whereas reallocations from MVPA to other behaviors showed unfavorable associations. Reallocations from LPA to sedentary behavior were associated with better physical health–related QoL and vice versa. Conclusion: Results reinforce the importance of MVPA for physical health but do not suggest that replacing sedentary behavior with LPA is beneficial for health and well-being.