Sensor fusion models in autonomous vehicles (AV) are used to compensate for inaccuracies in individual sensors. However, these models are not designed with security in mind. Recent works have demonstrated that physical attacks such as light injection or electromagnetic interference on a single sensor can compromise sensor fusion models, leading to critical consequences on the driving decision of the AV. This is because fusion models assume the integrity of the sensor data. This assumption is broken under physical attacks, leading to incorrect and unsafe decisions from the fusion model. Addressing the security of fusion models requires a systemic approach involving redundancy, anomaly detection, temporal and contextual scene understanding, and real-time dynamic adjustments to sensor behaviors. As sensor fusion models become increasingly popular, let’s define challenges and strategic measures from the intersection between physics and computer science to safeguard them against sensor attacks.