Background: Epilepsy and multiple sclerosis (MS) are two neurological diseases known to greatly influence a patient's life. The primary aim of this study was to describe the prevalence of sexual dysfunction in patients with epilepsy and MS and investigate whether there is an association between disease, sexual function, and physical and mental health. A secondary aim was to investigate whether there is a difference in sexual function between patients with MS and epilepsy. Methods: A total of 414 patients were included in this descriptive cross-sectional study. Three patient report questionnaires were used for measurements: The Changes in Sexual Function Questionnaire (CSFQ) cut-off score; the Short Form 36 Health Survey (SF-36) divided into the Physical Component Summary (PCS) and Mental Component Summary (MCS), and the Life Satisfaction-11 (LiSat-11). Results: Patients with MS constituted 62% (n = 258) of the participants and patients with epilepsy 38% (n = 156). The prevalence of sexual dysfunction was 68% in women and 77% in men. No differences were found between patients with MS and epilepsy (p = 0.184), except for the CSFQ desire domain, as patients with epilepsy more often had a desire problem (p = 0.029). On the SF-36, patients with MS scored significantly worse on the PCS (p = 0.000). Patients with epilepsy scored significantly worse on the MCS (p = 0.002). No significant differences were found on the LiSat-11. Regression analysis with CSFQ as the dependent variable showed an association with the PCS in men and an association with both PCS and MCS in women. Conclusions: In this study, the cohort of patients with MS and epilepsy had negatively affected sexual function. The only significant difference between patients with MS and epilepsy in sexual function measured by the CSFQ-14, was found in the frequency of desire, in which a larger number of patients with epilepsy reported sexual dysfunction. In the studied cohort, sexual function in women is associated with both physical and mental health, and in men with physical health. These results should be considered when caring for patients with epilepsy and MS.