Object detection in mixed illumination and adverse conditions requires robust fusion of visible and thermal cues. The proposed Dual-Path Attention Fusion Module (DPAFM) performs adaptive integration of RGB and thermal features using a cross-modal interaction mechanism controlled by learnable gated attention. This module is designed to mitigate modality-specific noise and to suppress feature collapse that occurs under glare, low light, or other degradations.
DPAFM avoids the computational burden associated with dense global Transformer or Mamba blocks by applying an efficient gated attention scheme. The result is selective emphasis of reliable modality responses while down-weighting degraded channels, enabling the downstream stages to operate on cleaner, more discriminative multimodal feature representations.
After initial fusion, the framework constructs a sparse graph where detection candidates are modeled as nodes. The Hierarchical Attention Graph Neural Network (HA-GNN) carries out relational reasoning over this graph using decoupled appearance and spatial attention mechanisms. Appearance attention captures semantic similarity between candidate nodes, while spatial attention encodes topological and positional relationships.
This hierarchical, decoupled design allows weakened or missing local cues to be compensated by neighboring nodes: degraded nodes can "borrow" semantic strength from contextually related candidates. By limiting reasoning to candidate nodes rather than dense scene-wide tokens, HA-GNN provides a targeted contextual verification stage that reduces false negatives caused by local feature annihilation and improves precision in cluttered or degraded scenes.
The architecture draws on insights from visual neuroscience: the separation of ventral ("what") and dorsal ("where") processing streams motivates the decoupling of appearance and spatial attention in HA-GNN. The approach also leverages Gestalt-like completion principles, using relational dependencies across the visual field to reconstruct missing or weak local signals. These neurocomputational concepts inform both the cross-modal gating in DPAFM and the graph construction and attention hierarchy in HA-GNN.
The authors validate their framework on multiple publicly available multispectral datasets. Experiments were performed on the KAIST Multispectral Pedestrian Dataset and the R-LiViT dataset, with additional references to the FLIR Thermal Dataset as part of the study's data sources. Evaluations emphasize day and night performance, small-object detection, and runtime efficiency.
The implementation and processed result data supporting the findings are available on GitHub, and a minimal replication dataset is deposited on Zenodo. Raw source datasets (KAIST, R-LiViT, FLIR) are cited as public domain resources used for analysis.
The proposed two-stage framework demonstrably increases robustness to illumination-induced degradation and sensor-specific noise. Reported experimental outcomes include a reduction of the day–night performance gap from 72.4% to 7.4% and a substantial improvement in small-object detection, quantified in the study as a 275% increase. The method is also reported to run in real time at 27.4 FPS, indicating a practical balance between accuracy and computational efficiency for autonomous driving perception in challenging environments.
By combining adaptive fusion (DPAFM) with sparse hierarchical graph reasoning (HA-GNN), the framework improves detection of distant or low-response objects that conventional single-stage or dense-fusion methods miss.
The authors have made code and processed results publicly available via a GitHub repository. A minimal dataset required to replicate the results has been uploaded to Zenodo with a DOI provided in the source. Raw datasets used in experiments are publicly accessible: KAIST Multispectral Pedestrian Dataset, R-LiViT Dataset, and FLIR Thermal Dataset. The study declares no specific external funding and no competing interests.
The work emphasizes a decoupled two-stage design, an efficient graph construction strategy, and a tailored cross-modal interaction mechanism as advantages over dense Transformer- or Mamba-based approaches. While the paper reports large relative gains in day/night robustness and small-object detection, readers should consult the original article for full experimental details, ablation studies, and exact evaluation metrics. The architecture provides a neuro-inspired, generalizable paradigm for bridging local feature degradation and contextual verification, offering a favorable trade-off between detection performance and runtime for all-weather autonomous perception applications.