SleepFM: Decoding Systemic Physiology Through Multimodal Sleep Representations

Lloyd Price
4 hours ago
10 min read

Executive Summary

Sleep represents a dynamic, highly regulated biological state during which the human body undergoes continuous homeostatic adjustment across multiple organ systems. While point-in-time clinical assessments conducted during waking hours provide limited snapshots of physiological function, overnight polysomnography (PSG) captures continuous, multi-system signals across an uninterrupted six to eight hour window. Developed by researchers at Stanford University in collaboration with the Technical University of Denmark, Rigshospitalet and affiliated institutions,

SleepFM is a multimodal artificial intelligence foundation model engineered to derive latent representations of human physiology directly from polysomnographic recordings.

Trained on a curated dataset exceeding 585,000 hours of PSG recordings from approximately 65,000 individuals, SleepFM establishes a novel paradigm in computational medicine. By shifting away from narrow, single-task deep learning models toward scalable, self-supervised pretraining, the model extracts high-dimensional physiological embeddings that accurately forecast long-term health trajectories. Across systematic evaluations against electronic health record (EHR) outcomes, SleepFM demonstrated the ability to predict the future onset of 130 distinct disease categories, spanning neurodegenerative, cardiovascular, oncological, metabolic, and psychiatric conditions, with a Concordance Index and Area.

Under the Receiver Operating Characteristic curve of at least 0.75. These findings indicate that overnight physiological signals encode pervasive biomarkers of systemic health, framing sleep not merely as a localised neurological recovery phase, but as an expansive diagnostic window into underlying homeostatic resilience.

Computational Architecture and Representation Learning

Polysomnography presents substantial computational challenges due to variations in sensor hardware, channel configurations (montages), signal to noise ratios and complex multi-scale temporal dynamics. SleepFM addresses these limitations through a channel-agnostic, multimodal representation learning framework designed to process diverse physiological time-series without requiring standardized channel counts or spatial electrode orderings.

Signal Pipeline and Architecture Strategy

The raw input data processed by SleepFM encompasses four primary physiological signal modalities:

Brain Activity Signals (BAS): Electroencephalography (EEG) and Electrooculography (EOG) tracking cortical rhythms and eye movements.
Cardiovascular Signals: Electrocardiography (ECG) capturing cardiac electrophysiology and heart rate variability dynamics.
Respiratory Signals: Nasal/oral airflow, pulse oximetry, and chest or abdominal respiratory effort belts.
Electromyography (EMG): Muscle tone and limb movement channels.

All continuous signals are resampled to a standardised frequency of 128 and segmented into 5 second temporal windows, which serve as the model's fundamental input tokens. Feature extraction within each modality is executed by one-dimensional Convolutional Neural Network (1D-CNN) backbone encoders based on EfficientNet architectures. To accommodate montage heterogeneity across clinical cohorts, SleepFM employs a channel-agnostic attention-pooling layer that dynamically aggregates feature maps across available channels within a given modality.

Following intra-modality channel aggregation, a temporal Transformer block processes sequential token embeddings over a 5 minute context window, capturing short- and medium term temporal dependencies such as transient micro-arousals, sleep spindle bursts and respiratory events. For participant-level prediction tasks, an additional temporal pooling layer compresses all token embeddings across an entire overnight study into a single, unified 128-dimensional latent vector.

The Leave One Out Contrastive Learning Objective

The foundational technical advance of SleepFM is its self-supervised pre training objective: Leave One Out Contrastive Learning (LOO-CL). Traditional multi-view contrastive learning methods rely on pairwise alignment, contrasting single pairs of modalities (such as EEG versus ECG) independently. However, pairwise contrastive loss fails to capture higher-order interactions that naturally occur across all simultaneously recorded organ systems.

By challenging the model to reconstruct the latent state of an omitted physiological stream using the combined context of all other active streams, LOO-CL forces the network to learn unified cross-modal semantics. This strategy makes the learned representations resilient to missing channels or sensor artifacts during deployment while ensuring that the latent space captures holistic systemic interaction rather than isolated signal traits.

Pretraining Data Diversity and Cohort Integration

To achieve broad demographic and clinical generalisability, SleepFM was pretrained on a large-scale, multi-centre dataset comprising over 585,000 hours of PSG data from approximately 65,000 participants. The training corpus aggregates data from tertiary academic sleep centres, decentralised clinical trials and prospective epidemiological studies.

Cohort Name	Primary Clinical Setting & Population	Sample Size (N)	Specific Function in Model Lifecycle
Stanford Sleep Clinic (SSC)	Tertiary Academic Clinical Cohort (Ages 2–96; Recorded 1999–2024)	35,052 PSGs	Pretraining, Secondary Fine-Tuning, & Longitudinal EHR Linkage
BioSerenity	Decentralized / In-Home Clinical Diagnostic Studies	18,900 PSGs	Pretraining (Cross-Site Sensor & Hardware Heterogeneity)
Outcomes of Sleep Disorders in Older Men (MrOS)	Prospective Epidemiological Cohort (Community-Dwelling Older Males)	3,930 PSGs	Pretraining (Geriatric & Degenerative Baseline Dynamics)
Multi-Ethnic Study of Atherosclerosis (MESA)	Prospective Epidemiological Multi-Center Study	2,237 PSGs	Pretraining (Ethnic, Demographic, & Subclinical Cardiovascular Diversity)
Sleep Heart Health Study (SHHS)	Multi-Center Epidemiological Community Study (Ages $\ge 40$)	6,441 PSGs	Fully Held-Out External Dataset for Transfer & Generalization Validation

Quantitative Disease Risk Forecasting and Predictive Efficacy

To evaluate SleepFM's capacity to forecast long-term health outcomes, participant representations derived from baseline overnight PSG studies at the Stanford Sleep Clinic were linked to longitudinal Electronic Health Records. Diagnostic codes (ICD-9 and ICD-10) were mapped into 1,868 standardised clinical phecodes.

To enforce predictive rigour and prevent diagnostic contamination, ensuring the algorithm was forecasting incident conditions rather than detecting pre-existing diagnoses, researchers excluded all disease labels that occurred prior to or within 7 days immediately following the baseline sleep study. Fine-tuning for long-term time to event outcomes was conducted using lightweight neural network heads optimised with a multi label Cox proportional hazards loss.

SleepFM evaluated more than 1,000 disease categories in patient health records and identified 130 conditions that could be predicted with high accuracy, achieving a Bonferroni-corrected P < 0.01.

Disease Category / Phenotype Condition	SleepFM C-Index (95% CI)	SleepFM 6-Year AUROC	Demographics Baseline C-Index	End-to-End Supervised PSG Baseline
Senile Dementia	0.99	> 0.85	0.87	< 0.85
Parkinson’s Disease	0.89	0.93	< 0.75	< 0.80
Prostate Cancer	0.89	> 0.85	< 0.75	< 0.75
Breast Cancer	0.87	> 0.80	< 0.75	< 0.75
All-Cause Mortality	0.84 (0.81–0.87)	0.85	0.78	0.78
Dementia (All Types)	0.85 (0.82–0.87)	> 0.85	0.78	< 0.78
Hypertensive Heart Disease	0.84	> 0.80	< 0.75	< 0.75
Mild Cognitive Impairment (MCI)	> 0.80	0.84	< 0.70	< 0.72
Atherosclerosis	0.92	> 0.85	0.74	< 0.80
Myocardial Infarction (MI)	0.81	> 0.80	< 0.74	< 0.75
Myoneural Disorders	0.81	> 0.80	0.42	< 0.60
Heart Failure	0.80 (0.77–0.83)	> 0.78	< 0.72	< 0.73
Developmental Delays and Disorders	0.80	0.84	0.58	< 0.65
Chronic Kidney Disease (CKD)	0.79 (0.77–0.81)	> 0.75	< 0.72	< 0.72
Stroke	0.78 (0.76–0.81)	> 0.75	< 0.71	< 0.71
Atrial Fibrillation	0.78	> 0.75	< 0.70	< 0.72

SleepFM consistently outperformed two primary baseline models across disease categories: a demographics-only model (a multilayer perceptron trained on age, sex, body mass index, and race/ethnicity) and an end to end supervised PSG model trained directly from raw signals without self-supervised pre-training. Relative improvements in AUROC ranged from 5% to 17% over these baseline models, with the most pronounced gains observed in neurological, haematological and neuromuscular categories.

Benchmark Validation and Sample Efficiency

To confirm that a foundation model trained for multi-disease prediction retains strong diagnostic accuracy on standard clinical tasks, SleepFM was benchmarked against specialised single-task architectures for sleep staging and Sleep-Disordered Breathing (SDB) detection.

Clinical Diagnostic Benchmark Task	Primary Metric	SleepFM Performance	Specialised Benchmark Baseline Models
Sleep Staging (5-Stage Classification)	Mean F1-Score	0.70 – 0.78	0.70 – 0.78 (U-Sleep, YASA, GSSC, STAGES)
Sleep Staging Discrimination	Macro AUROC	0.906	0.842 (End-to-End Supervised CNN)
Sleep Staging Precision-Recall	Macro AUPRC	0.685 [	0.579 (End-to-End Supervised CNN)
Sleep Apnea Detection (Presence)	Accuracy / AUROC	0.87 / 0.90– 0.94	0.843 (AUROC End-to-End CNN)
Sleep Apnea Severity Classification	Accuracy / AUPRC	0.69 / 0.711	0.555 (AUPRC End-to-End CNN)

SleepFM matched or exceeded the performance of task-specific architectures like U-Sleep, YASA, GSSC, and STAGES on traditional tasks while providing broader health risk assessment capabilities.

A key advantage of self-supervised foundation models is label efficiency during downstream task adaptation. When fine-tuned on restricted data subsets from the Stanford cohort, SleepFM trained on only 10% of available labeled data outperformed the demographics-only baseline trained on 100% of the data across all evaluated disease categories.

Furthermore, testing on the held-out Sleep Heart Health Study (SHHS) demonstrated out-of-distribution transferability, maintaining predictive accuracy for cardiovascular death, stroke and heart failure despite differences in clinical environments and hardware configurations.

Physiological Mechanisms and Systemic Desynchrony

The predictive capability of SleepFM across 130 disease categories offers insights into human sleep physiology. Conventional sleep medicine relies heavily on macro-structural metrics: total sleep time, sleep efficiency, epoch-based sleep staging and the Apnea-Hypopnea Index (AHI).

However, SleepFM extracts fine-grained physiological signals distributed across multiple organ systems.

Cross System Desynchrony as an Early Biomarker

A key finding from ablation studies of SleepFM is that single-modality representations (such as ECG alone or EEG alone) exhibit lower accuracy in long-term disease prediction compared to the full multimodal model. The highest predictive signal emerges from contrasting physiological streams against one another.

In healthy individuals, the central nervous, cardiovascular, respiratory and neuromuscular systems maintain functional coordination during NREM and REM sleep stages.

SleepFM detects subtle patterns of physiological de-synchrony, instances where cortical activity displays NREM delta rhythms while cardiac electrophysiology exhibits elevated sympathetic arousal characteristic of wakefulness. This neuro-cardiac or neuro-respiratory decoupling acts as an early marker of homeostatic instability, signalling autonomic dysfunction, chronic vascular inflammation, or subclinical brainstem injury years before overt clinical symptoms appear.

Polysomnography as an Unconscious Stress Test

In waking life, voluntary movement, cognitive compensation, and homeostatic buffering mechanisms often obscure underlying organ system fragility. During sleep, conscious behavioural compensations are suspended, placing the body under an uninterrupted multi-hour physiological stress test.

Overnight PSG records continuous dynamic responses to transient stressors, including subtle oxygen desaturations, micro-arousals, heart rate decelerations and motor fluctuations. SleepFM's 128-dimensional latent vector integrates these recurrent micro-stressors into a unified representation. For example, the model forecasts myocardial infarction and heart failure not merely by identifying overt arrhythmias, but by capturing altered heart rate variability patterns during transient arousal states, subclinical hypoxic events, and impaired autonomic recovery during NREM sleep.

Micro-Structural Signatures in Neurodegeneration

Neurological disorders demonstrated high predictive accuracy in SleepFM, particularly Parkinson's disease and senile dementia. Neurodegenerative proteinopathies (such as alpha synucleinopathies and tauopathies) frequently damage subcortical brainstem centres, including the locus coeruleus, pedunculopontine nucleus and sublaterodorsal nucleus, years before cognitive or motor impairments manifest clinically.

Because these subcortical structures regulate sleep architecture, REM muscle atonia and autonomic tone, early neurodegenerative changes alter overnight physiology. SleepFM identifies subtle disruptions in REM muscle tone, electroencephalographic spindle degradation and micro-fragmentation patterns that escape visual inspection during routine manual scoring, providing a non-invasive approach for early neurodegenerative risk profiling.

Translational Horizons, Ethical Paradigms and Future Outlook

Translating multimodal sleep foundation models into routine clinical practice involves operational, computational and ethical considerations.

Translational & Operational Domain	Key Technical Challenges	Strategic Implementation Paradigms
Consumer Wearable Adaptation	Signal loss moving from 12-channel clinical PSG to single-lead ECG, PPG, or accelerometry.	Leveraging Leave One Out embeddings to reconstruct missing modalities; shifting from single-night lab studies to continuous longitudinal tracking.
Algorithmic Interpretability	High-dimensional latent vectors (128-D) lack direct natural language explanations.	Applying post-hoc attribution methods and channel saliency mapping to connect vector patterns back to clinical physiological features.
Ethical & Governance Frameworks	Risk of consumer anxiety, over-testing, and improper health risk interpretation.	Establishing clinical decision support pathways that separate wellness guidance, risk screening, and formal medical diagnosis.

Wearable Technology Integration

While SleepFM was pre-trained on clinical-grade PSG recordings involving full EEG, ECG and respiratory channels, widespread health screening requires deployment beyond dedicated sleep laboratories. The channel agnostic design established by the LOO-CL training strategy directly supports this transition.

Consumer wearable devices, such as smartwatches, rings, and chest patches, typically capture reduced signal modalities, such as photoplethysmography (PPG), single-lead ECG and peripheral pulse oximetry.

Because SleepFM was trained to reconstruct omitted modalities using available channels, its core embeddings can be fine-tuned to lower density consumer sensor inputs. This flexibility enables a transition from point in time clinical sleep studies to continuous home monitoring, allowing models to track changes in a patient's baseline risk trajectory over extended periods.

Model Explainability and Clinical Validation

A primary barrier to clinical integration is model interpretability. While SleepFM demonstrates high predictive accuracy for conditions like breast cancer and chronic kidney disease, the network does not natively output explanations in natural language.

Clinical deployment requires explainability tools, such as gradient-based feature attribution, channel-saliency analysis and counterfactual signal synthesis. Clinicians must be able to link high-risk predictions to recognisable physiological phenomena, such as specific nocturnal hypoxic loads, altered autonomic recovery, or electroencephalographic micro-arousals, before initiating preventative interventions or further diagnostic workups.

Ethical Governance: Screening vs. Diagnosis

The ability to predict 130 medical conditions from overnight sleep data necessitates clear boundaries regarding clinical utility. Healthcare systems and industry partners must maintain clear distinctions between general wellness guidance, automated risk screening and definitive clinical diagnosis.

Unregulated delivery of long-term disease risk scores directly to consumers could induce unnecessary distress or lead to clinical over-utilisation. Responsible deployment of models like SleepFM requires structured decision support frameworks that route predictions through qualified healthcare providers, ensuring risk scores are contextualised alongside clinical history, targeted diagnostics, and actionable preventive care.

Conclusions

SleepFM demonstrates that multimodal AI foundation models can learn complex physiological representations directly from polysomnography recordings. By applying self-supervised Leave One Out Contrastive Learning across more than 585,000 hours of clinical sleep data, the model transforms overnight physiological signals into versatile representations capable of predicting long-term disease trajectories.

Predicting future risk across 130 health conditions underscores that sleep serves as a rich window into multi-system homeostatic function. As these foundation models adapt to consumer wearable technologies and longitudinal health monitoring, AI-driven physiological analysis offers a non-invasive path toward early disease detection, proactive health management, and personalised preventive care.

Nelson Advisors > European HealthTech, MedTech, Digital Health Investment Banking

Nelson Advisors specialise in Mergers and Acquisitions, Partnerships and Investments for Digital Health, HealthTech, MedTech, Health IT, Consumer HealthTech, Healthcare Cybersecurity, Healthcare AI companies. www.nelsonadvisors.co.uk

Follow Nelson Advisors LinkedIn Page > https://www.linkedin.com/company/nelson-advisors/

Nelson Advisors regularly publish Thought Leadership articles covering market insights, industry trends, deal commentary, market analysis & predictions. https://www.healthcare.digital

Nelson Advisors publish Europe's Leading Healthcare Technology Investment Banking Newsletter every week, join 5000+ HealthTech and MedTech subscribers today! https://lnkd.in/e5hTp_xb

#VentureCapital #PrivateEquity #Founders #SeriesA #SeriesB #Founders #SellSide #TechAssets #Fundraising #BuildBuyPartner #GoToMarket #PharmaTech #BioTech #Genomics #NelsonAdvisors #HealthTech #MedTech #DigitalHealth #HealthIT #Cybersecurity #HealthcareAI #FemTech #Mergers #Acquisitions #Partnerships #Growth #Strategy #NHS #UK #Europe #USA #Canada