This longitudinal multicenter study integrated a photo function and written instructions into the Swiss Clinical Quality Management in Rheumatic Diseases (SCQM) registry and the mySCQM mobile application to enable both patient- and health-care-professional (HCP)-collected hand photographs for remote monitoring of inflammatory arthritis.
The study sought to evaluate practicality, image quality, patient engagement, and the clinical utility of an automated, machine learning-based digital biomarker — the finger fold index (FFI) — derived from dorsal hand photographs to detect proximal interphalangeal (PIP) joint swelling in patients with rheumatoid arthritis (RA) and psoriatic arthritis (PsA).
Between 2023 and 2025, 374 patients with RA or PsA were enrolled. HCPs captured 977 hand images during routine visits, and 174 patients used the mySCQM app to submit a total of 1,228 hand images after receiving instructions. Patients continued to contribute images longitudinally, demonstrating sustained engagement.
After manual quality assessment and subsequent automated processing, a final dataset of 1,729 hand images representing 4,048 PIP joints was available for analysis.
Image quality suitable for execution of the ML pipeline was comparable between acquisition sources: 78.3% of patient-acquired images and 73.5% of HCP-acquired images met the criteria to run the model. Patients contributed a mean of seven images over the data collection period, indicating ongoing engagement following instruction.
However, a substantial fraction of individual cropped joints (23.1%) had to be removed after running the FFI algorithm due to false detections, commonly caused by incorrect hand positioning or other acquisition errors.
Photographs that passed manual quality control were processed by an automated computer vision pipeline to detect dorsal finger folds and PIP joint diameters and to compute the finger fold index (FFI) — a ratio of proximal interphalangeal joint diameter to the surface area of dorsal finger folds. The FFI is presented as an image-derived digital biomarker intended to reflect joint swelling.
Associations between the FFI and clinical PIP joint swelling, as well as with disease activity scores (DAS28-CRP and RADAI-5), were evaluated. The study used correlation analyses for cross-sectional comparisons and assessed longitudinal concordance between change in clinical swelling and change (delta) in FFI at follow-up visits. A generalized linear mixed model (GLMM) was employed to estimate associations between FFI and swelling while accounting for repeated measures and within-subject correlations.
After quality control, 1,729 images covering 4,048 PIP joints were analyzed. Image suitability for running the algorithm was similar for patient and HCP images, though nearly one-quarter of cropped joints were excluded post-processing due to detection errors.
In HCP-acquired images, mean FFI and DAS28-CRP showed a weak but statistically significant Spearman correlation (ρ = 0.164; 95% CI [0.004 to 0.317]; p = 0.039). Mean FFI did not correlate with RADAI-5 scores in RA patients (r = 0.007; p = 0.932; 95% CI [-0.169 to 0.183]).
For longitudinal changes, clinical swelling resolved in 40 joints; in 68.0% of these, the direction of the delta FFI agreed with the clinical change. Among 13 joints that developed incident swelling, 87.5% had concordant direction of delta FFI.
The GLMM, which accounted for repeated measures and within-subject correlations, found no significant associations between swelling and either joint location or time-varying FFI. The model also did not find evidence of interaction between FFI and specific PIP joint location.
Although image quality was comparable between patient- and HCP-acquired photographs, the study reports that 23.1% of cropped joints were unusable after ML processing due to incorrect diameter or finger fold detection, often related to suboptimal hand positioning. The GLMM did not confirm an association between swelling and FFI when modeling repeated measures, indicating limited robustness in some analyses.
The authors highlight the need for more standardized image acquisition protocols and further refinement of the FFI algorithm to reduce the proportion of images where the biomarker cannot be applied.
Integration of patient self-imaging into a remote monitoring app for inflammatory arthritis is feasible and achieved image quality comparable to clinician-acquired images in this real-world cohort. The FFI derived from collected images showed associations with clinical PIP joint swelling and with DAS28-CRP in HCP images, but not with the RADAI-5 patient-reported score. A notable proportion of images could not be processed successfully by the algorithm, underscoring the need for improved acquisition standardization and algorithmic development before routine clinical application.
The study was approved by the Commission cantonale d'ethique de la recherche sur l'etre humain (CER-VD) (2020-00033) and was conducted in accordance with the Declaration of Helsinki. Competing interests reported include MB and TH as shareholders of Atreon SA, and TH having filed a patent jointly with CHUV related to the image analysis for monitoring swelling of body parts. The manuscript is a preprint and has not been peer reviewed.