Weather-Robust Multi-Modal Fusion for Autonomous Driving: Reliability-Aware Attention in Adverse Environments

Weather-Robust Multi-Modal Fusion for Autonomous Driving: Reliability-Aware Attention in Adverse Environments

Reza Homayouni Amlashi1

1) Department of Computer Engineering, Islamic Azad University Babol Branch, Babol, Iran

IOI Code : ACEF-ACAZDZ
Publication : 6th. International Congress On Engineering, Technology and Applied Sciences(etas2027.com)
Abstract :
    Autonomous driving perception systems depend upon types of sensors. These include LiDAR, RGB cameras, mmWave Radar and Gated Near-Infrared (NIR). The goal is to detect objects in three dimensions. Bad weather for example fog makes sensors fail in different ways. LiDAR suffers from backscatter and loss of points. Cameras lose a lot of contrast. Radar and Gated NIR stay fairly reliable. Current methods that combine sensors usually assume that every sensor works the same. They also rely heavily on LiDAR as a reference for geometry. This causes the system to perform poorly when the weather is bad. In this paper we introduce Weather‑Robust Fusion (WRF). WRF is a system that uses a Multi‑Modal Latent Attention (MMLA) module at its core. MMLA treats fusion of modalities as a process that changes reliability. It lets sensors—Radar, Camera and NIR—compete for attention against queries that come from LiDAR. All of this happens in a shared space. If the main sensor degrades the attention weights automatically move toward sensors that re more reliable. This happens without needing to classify the weather. We also add a training trick called an exact inverse augmentation strategy. This keeps the geometry consistent among all sensors while the system learns. We ran tests on bad‑weather benchmarks. WRF reaches 89.8% mAP at 0.5 under weather. When fog is severe and LiDAR points drop out gradually the model stays robust. Even when LiDAR is completely off WRF keeps 61.3% mAP at 0.5. This is much better, than multi‑modal baselines.

Keywords : autonomous driving multi modal fusion adverse weather perception bird’s eye view (bev) cross attention robust 3d detection