Schizophrenia and bipolar disorder are highly heritable severe mental disorders associated with social impairments. Moreover, partners to individuals with one of these disorders display poorer functioning and more psychopathology, but their social skills and the transgenerational transmission remains uninvestigated. Therefore, we aimed to examine social responsiveness in families with parental schizophrenia or bipolar disorder. The cohort consists of 11-year-old children with at least one parent with schizophrenia (n = 179) or bipolar disorder (n = 105) and population-based controls (PBC, n = 181). Children and parents were assessed with The Social Responsiveness Scale, Second Edition. Duration of time each parent and child have lived together was ascertained through interviews. Parents with schizophrenia and parents with bipolar disorder exhibited poorer social responsiveness compared with PBC parents. Parents with schizophrenia displayed poorer social responsiveness compared with parents with bipolar disorder. Schizophrenia co-parents exhibited poorer social responsiveness compared with bipolar co-parents and PBC co-parents. We found significant positive associations between parents’ and children's social responsiveness, with no interaction effect of duration of time living together. Considering that social impairments are suggested as a vulnerability marker, this knowledge calls for increased attention towards vulnerable families, particularly those where both parents have social impairments.