Purpose: To investigate whether health literacy (HL) among informal caregivers of breast cancer (BC) survivors is associated with patient psychological outcomes. Methods: We used data (n = 340 pairs) from baseline questionnaires administered in the MyHealth trial investigating nurse-led BC follow-up. All BC survivors and their invited caregivers were included immediately after completion of primary treatment. We performed multivariate regression analyses to examine the association between caregiver HL (nine dimensions as measured by the Health Literacy Questionnaire) as exposure and patient depression, anxiety, and health-related quality of life (HQoL) as outcomes. We further examined whether any association differed according to type of caregiver, patient HL, and patient activation (skill in managing one’s health). Results: Three dimensions, “ability to engage with providers” (β = − 0.2), “navigating the system” (β = − 0.2), and “understand health information” (β = − 0.2), were significantly associated with lower patient depression (p < 0.05), while four dimensions, “having sufficient information” (β = 0.3), “navigating the system” (β = 0.2), “find health information” (β = 0.2), and “understand health information” (β = 0.2), were significantly associated with better patient HQoL (p < 0.05). No significant associations were found for anxiety. Patient HL and activation did not significantly modify the associations, while certain associations for depression were stronger in patients with non-partner caregivers. Conclusions: The HL of informal caregivers may play an important role in optimizing psychological outcomes in cancer survivors. Implications for Cancer Survivors: Involving informal caregivers, who can provide support related to health information and services, may be beneficial for the psychological well-being of cancer survivors.