This study examines depression-associated speech representations in individuals with Alzheimer's disease (AD) using an explainable multimodal computational framework. The approach integrates linguistic and acoustic features derived from elicited picture-description speech. Because the AD cohort lacked directly observed depression labels, the authors used a source-informed cross-dataset transfer strategy: a depression-labelled source corpus informed latent-space comparisons with an AD speech corpus. Unsupervised clustering on the AD representations identified two distinct speech sub-phenotypes; one cluster was geometrically proximate to the depression-labelled source centroid when measured by cosine similarity in a shared latent space. A retained multimodal artificial neural network (ANN) achieved 92.73% separability accuracy between the two clusters on an internal hold-out test set. Interpretability analyses used Shapley Additive exPlanations (SHAP) to attribute influential acoustic and linguistic features and to organise them into a multimodal taxonomy. The authors additionally propose a Multimodal SHAP Attribution Magnitude (MSAM) as an exploratory model-based attribution score. Importantly, the preprint emphasises that latent-space proximity and model separability do not establish clinical validity, longitudinal stability, or diagnostic validity of the identified sub-phenotypes.
The analysis used two established speech corpora. The depression-labelled source dataset was the DAIC-WOZ corpus, accessed under a formal Data Use Agreement with the University of Southern California. The target AD corpus was DementiaBank, accessed via the TalkBank repository under their protocols for affiliated researchers. The study is a secondary analysis of retrospective, controlled-access datasets; no new primary data collection took place. The authors state that the original data collections were approved by the respective institutional review boards and that informed consent was obtained by the original investigators. All datasets used were de-identified prior to analysis, and ethical restrictions prevented depositing raw audio in unrestricted public repositories.
The framework integrates two modalities: linguistic features derived from transcribed picture-description speech and acoustic features derived from the corresponding audio. The authors report constructing a shared latent representation that fuses these linguistic and acoustic inputs, enabling joint analyses and cross-dataset comparisons. Specific feature lists, exact preprocessing steps, model architectures or hyperparameters were not reported in detail in the source abstract; the full manuscript or supplementary materials would need to be consulted for those technical specifics.
To address the absence of observed depression labels in the AD cohort, the authors implemented a source-informed cross-dataset transfer strategy. A shared latent space was learned such that representations from the depression-labelled source corpus (DAIC-WOZ) and the AD target corpus (DementiaBank) could be compared. The geometric relationship in that shared latent space was assessed using cosine similarity. The authors explicitly note that proximity to a depression centroid in latent space reflects geometric similarity and may capture overlapping domain attributes as well as affective variance, cautioning against equating geometric proximity with clinical diagnosis.
Unsupervised clustering applied to the AD speech representations revealed two distinct clusters, described by the authors as speech sub-phenotypes within the AD cohort. One of these clusters demonstrated mathematical proximity to the depression-labelled source centroid in the shared latent space. The preprint frames these clusters as computationally discovered groupings that merit further clinical validation; the study does not assert that these clusters correspond to clinically validated depressive states.
To validate the structural separability and learnability of the unsupervised partition, the authors trained a retained multimodal ANN in a proxy supervised classification task to distinguish the two clusters. The ANN combined information from both linguistic and acoustic modalities. On an independent internal hold-out test set, the model achieved 92.73% cluster separability accuracy. The authors emphasise that this high accuracy demonstrates the clusters were highly learnable within the dataset and partitioning but does not establish clinical validity, longitudinal stability, or diagnostic utility of the clusters.
Interpretability analyses used Shapley Additive exPlanations (SHAP) to extract feature attributions from the multimodal model and to identify which acoustic and linguistic features contributed most to the learned representations. The authors organised influential features into a multimodal taxonomy based on SHAP attributions. They also derived a Multimodal SHAP Attribution Magnitude (MSAM) score as an exploratory, model-derived attribution magnitude; the MSAM is described as an attribution metric rather than a clinically validated measure of depression severity.
The authors explicitly caution that geometric proximity in latent space and model separability do not equate to clinical diagnosis. They note the absence of directly observed depression labels in the AD cohort and clarify that their cross-dataset transfer strategy seeks structural correspondences rather than clinical confirmation. The study establishes an interpretable computational baseline and structural foundation intended to support future prospective validation in independently labelled clinical cohorts. The preprint does not report longitudinal stability, external diagnostic validation, or clinical outcome correlations for the discovered sub-phenotypes.
The DAIC-WOZ dataset is available to researchers upon signing a Data Use Agreement via the University of Southern California. DementiaBank is accessible to affiliated researchers through TalkBank. Raw audio files cannot be deposited in unrestricted public repositories due to ethical restrictions on sensitive human speech recordings. The authors state that all data were processed and stored in compliance with the governing ethical restrictions and that no raw audio data have been released without controlled access.