The serotonergic system integrates sex steroid information and plays a central role in mood and stress regulation, cognition, appetite and sleep. This interplay may be critical for likelihood of developing depressive episodes, at least in a subgroup of sensitive individuals. The serotonin 4 receptor (5-HT₄R) indexes central serotonergic tonus, which may be related to endogenous sex-steroid levels in the mentally healthy state even though this remains elusive. Here we evaluate if peripheral levels of estradiol and testosterone are associated with 5-HT₄R binding as imaged by [¹¹C]SB207145 positron emission tomography in a group of 41 healthy men. We estimated global 5-HT₄R binding using a latent variable model framework, which models shared correlation between 5-HT₄R across multiple brain regions (hippocampus, amygdala, posterior and anterior cingulate, thalamus, pallidostriatum and neocortex). We tested whether testosterone and estradiol predict global 5-HT₄R, adjusting for age. We found that testosterone, but not estradiol, correlated negatively with global 5-HT₄R levels (p = 0.02) suggesting that men with high levels of testosterone have higher cerebral serotonergic tonus. Our findings corroborate the link between sex hormone levels and serotonin signalling. Future longitudinal studies in clinical relevant populations are needed to elucidate the potential importance of testosterone in the pathophysiology of e.g. major depression and its treatment.