Background: Despite the genetic overlap between bipolar disorder and schizophrenia, working memory impairments are mainly found in children of parents with schizophrenia. However, working memory impairments are characterized by substantial heterogeneity, and it is unknown how this heterogeneity develops over time. We used a data-driven approach to assess working memory heterogeneity and longitudinal stability in children at familial high risk of schizophrenia (FHR-SZ) or bipolar disorder (FHR-BP). Methods: Based on the performances on four working memory tasks by 319 children (FHR-SZ, N = 202, FHR-BP, N = 118) measured at age 7 and 11, latent profile transition analysis was used to test for the presence of subgroups, and the stability of subgroup membership over time. Population-based controls (VIA 7, N = 200, VIA 11, N = 173) were included as a reference group. The working memory subgroups were compared based on caregiver- and teacher ratings of everyday working memory function, and dimensional psychopathology. Results: A model with three subgroups characterized by different levels of working memory function (an impaired subgroup, a mixed subgroup, and an above average subgroup) best fitted the data. The impaired subgroup had the highest ratings of everyday working memory impairments and psychopathology. Overall, 98 % (N = 314) stayed in the same subgroup from age 7 to 11. Conclusion: Persistent working memory impairments are present in a subset of children at FHR-SZ and FHR-BP throughout middle childhood. Attention should be given to these children, as working memory impairments influence daily life, and may serve as a vulnerability marker of transition to severe mental illness.