Background: Breast cancer is the most frequently diagnosed cancer in the world. Exercise is widely recommended for patients with breast cancer during and after treatment. However, there is a lack of studies investigating barriers related to participation in real-world exercise-based trials for older patients with breast cancer. Objective: To explore reasons for declining participation in an exercise-based trial among older patients with breast cancer during (neo)adjuvant or palliative systemic treatment. Methods: A qualitative study using semi-structured interviews. Patients who declined participation in an exercise-based trial (N = 50) were invited to participate. Semi-structured interviews were conducted with 15 participants. Interviews were audio-recorded, transcribed verbatim and analyzed using thematic analysis. Results: Identified main themes: 1) Lack of energy and resources, including two subthemes: 1a) Overwhelmed both mentally and physically, and 1b) The program is too comprehensive; 2) Uncertainty about reactions to chemotherapy; 3) The hospital is not the optimal exercise setting, including two subthemes: 3a) Transportation and time consumption, and 3b) No desire to spend additional time at the hospital; and 4) Staying active in my own way, including two subthemes: 4a) Motivation to exercise, and 4b) Preferences for exercise activities. Conclusion: Many barriers were identified, including time of recruitment, information overload, symptoms and side effects, and the hospital as the exercise setting due to practical challenges and negative feelings. Participants were motivated to exercise from knowledge about the benefits of exercising. Furthermore, they preferred activities that they were already involved in or had experience with.