Microdialysis perfusion of 5‐HT into hypoglossal motor nucleus differentially modulates genioglossus activity across natural sleep‐wake states in rats

Serotonin (5-hydroxytryptamine, 5-HT) excites hypoglossal (XII) motoneurons in reduced preparations, and it has been suggested that withdrawal of 5-HT may underlie reduced genioglossus (GG) muscle activity in sleep. However, systemic administration of 5-HT agents in humans has limited effects on GG activity. Whether 5-HT applied directly to the XII motor nucleus increases GG activity in an intact preparation either awake or asleep has not been tested. 2. The aim of this study was to develop a novel freely behaving animal model for in vivo microdialysis of the XII motor nucleus across sleep-wake states, and test the hypothesis that 5-HT application will increase GG activity. 3. Eighteen rats were implanted with electroencephalogram and neck muscle electrodes to record sleep-wake states, and GG and diaphragm electrodes for respiratory muscle recording. Microdialysis probes were implanted into the XII motor nucleus and perfused with artificial cerebrospinal fluid (ACSF) or 10 mM 5-HT. 4. Normal decreases in GG activity occurred from wakefulness to non-rapid eye movement (non-REM) and REM sleep with ACSF (P < 0.01). Compared to ACSF, 5-HT caused marked GG activation across all sleep-wake states (increases of 91-251 %, P < 0.015). Importantly, 5-HT increased sleeping GG activity to normal waking levels for as long as 5-HT was applied (3-5 h). Despite tonic stimulation by 5-HT, periods of phasic GG suppression and excitation occurred in REM sleep compared with non-REM. 5. The results show that sleep-wake states differentially modulate GG responses to 5-HT at the XII motor nucleus. This animal model using in vivo microdialysis of the caudal medulla will enable the determination of neural mechanisms underlying pharyngeal motor control in natural sleep.