Imagine a vehicle moving along a dark, unlit road. A conventional camera barely discerns shapes, but its thermal sensor clearly traces the warm outline of a pedestrian about to step into the path. It’s the perfect argument in favor of this technology: where human sight falters, heat reveals a person’s presence.
Yet a study unveiled at the NDSS 2026 security symposium shows that this same advantage can turn against us. Under certain conditions, heat sources—natural or deliberately placed—can cause the camera to stop “seeing” the pedestrian, fabricate obstacles that do not exist, or retain ghost images for minutes. And strikingly, it need not require access to the vehicle’s software to provoke such effects.
The Paradox of the Sensor That Sees in the Dark
Everyday intuition holds that a human body, warmer than its surroundings, should appear as a bright spot against a cool background. That assumption underpins much of the trust placed in thermal cameras for nighttime driving, drone surveillance, and mobile robotics.
The problem is that a thermal sensor does not capture “people” directly: it captures infrared radiation and converts it into an image through several processing stages. That interpretive architecture is precisely where the Florida team, in collaboration with researchers from the University of California, Irvine, Toyota, and Japan’s University of Electro-Communications, identified a critical vulnerability.
To dispel a common confusion at the outset, this work does not analyze fences, blinds, or repetitive façade patterns, nor patterns that deceive the stereoscopic vision of ordinary cameras. It focuses exclusively on vulnerabilities of thermal cameras and on how heat alters what the system believes it is seeing. In other words, it is not the geometry of a scene that confuses the sensor, but the temperature.
Three Layers That Can Distort the Scene
Between the radiation that reaches the sensor and the detector’s final decision lie several processing steps. The study identifies three classes of vulnerability, one for each layer of interpretation, spanning a range of failures that echoes another biological question: why do living beings share one language to turn genes on, yet a thousand dialects to turn them off?
The first affects the equalization of the image, the step that translates thermal values into brightness or color levels. A high-heat source can shift the relative temperature range and compress the useful contrast, so that a pedestrian’s silhouette no longer stands out against the background. The sensor keeps receiving radiation, but the resulting image no longer faithfully represents the object that matters.
The second vulnerability concerns thermal calibration, the internal mechanisms that compensate for differences between pixels to maintain a stable device response. Under certain conditions, this process can generate patterns that a detector interprets as a real object or obstacle where nothing exists.
The third relates to image acquisition and the optics of the lens, which can produce reflections, phantom images, and persistent thermal artifacts. This outer layer—the first to touch the light—poses a question akin to other superficial elements: is the potato skin the healthiest part of the tuber, or is it a part worth peeling away? Here a troubling nuance appears: some of these alterations remained visible for several minutes after removing the heat source. In other words, the sensor preserves a misleading memory of something that is no longer present.
Accident, Manipulation, or Hacking: Three Very Distinct Avenues
One of the most valuable contributions of the work is to shift the discussion from “hacking” to perception engineering. The study implicitly distinguishes three risk pathways that should not be mixed.
The first is an accidental failure: hot surfaces, industrial machinery, reflections, or abrupt temperature shifts that are part of everyday surroundings and could distort the image without anyone intending it. The second is deliberate scene manipulation, when a heat source is placed with the intent of confusing the system. The third would be the classic cyberattack, with access to software or hardware.
The key, and at the same time the most delicate point, is that the first two avenues do not require breaking into the system. They affect perception from the outside, acting on the physical world that the sensor observes. In the interest of caution, we will not describe here how to reproduce the effect: the objective is to understand the vulnerability, not to enable its exploitation.
What the Figures Say, and, More Important, What They Don’t
The results are compelling, but must be interpreted with care. All figures come from a concrete experimental protocol, with specific cameras, detectors, and scenarios, and none imply a universal probability of failure for any vehicle or robot in real-world use.
The evaluation covered three thermal cameras (the FLIR Boson, InfiRay T2S, and FPV XK-C130), three object detectors, and two fusion models that combine visible and thermal imagery. Based on that, the main findings were as follows:
- Thermal alterations reduced up to 50% in average precision (mean average precision or mAP) for pedestrian detection.
- In models that fused visible and thermal images, the precision drop reached up to 45%.
- In certain tests conducted at speeds of up to 40 km/h, the systems failed to detect pedestrians in as many as 100% of cases.
- Heat sources generated false obstacles with a maximum success rate of 91%.
- Some artifacts persisted for several minutes after removing the heat source.
Let’s interpret these numbers with caution. The 100% and 91% are maxima observed in specific scenarios, not a forecast for “any autonomous car.” The team itself does not claim that a vehicle can be controlled easily, but rather points to a subtler, fundamentally important idea: an architecture of perception can err when it relies too heavily on a single sensory modality.
This nuance explains why the 45% decline in fusion models is so revealing. Combining visible and thermal signals seemed like the obvious solution, yet it did not eliminate the problem. If a model integrates both cues without estimating which is reliable at each moment, a distorted thermal input can contaminate the joint decision. Put simply, two images do not automatically equate to two independent observations.
Real Redundancy Is Not About Adding Sensors, But About Comparing Them
This is where defense comes into play, and where the study becomes more constructive. The researchers developed real-time signal processing techniques capable of detecting and suppressing certain deceptive thermal artifacts. It is a mitigation, not a guarantee: it cannot be assumed to work across all sensors, models, and weather conditions.
Beyond that specific countermeasure, the work points toward design principles that transcend the thermal case. The first is to combine sensors with different physical foundations, such as radar and lidar alongside conventional and thermal cameras, whenever feasible in use and cost, without falling into the trap of thinking that more data always equates to better perception. The advantage of such diversity lies not in accumulating inputs but in ensuring that a radio-wave–based sensor is not fooled by the same phenomena as an infrared one.
The second principle is to detect contradictions. A thermal interference that insists on marking an obstacle where radar and lidar see nothing should be given less trust, just as with other obstacle detectors that nature has already refined: they are not mere “whiskers” guiding every feline step, but rather part of a robust system where the sensors’ trust weights vary with their momentary reliability, rather than treating them as equally valid.
The third principle incorporates the temporal dimension that reveals artifact persistence. Validating a system cannot be limited to single frames: one must assess the stability of detections over time, the sensor’s memory, and its capacity to recover after a disturbance. An obstacle that lingers in the image for minutes after it has disappeared from the real world poses its own security concern.
The fourth, perhaps the most important, is to handle uncertainty. When signals from different sensors are incompatible and the system cannot confidently determine what it is seeing, prudent action is to reduce speed, brake, or adopt a safe maneuver once a pre-set uncertainty threshold is exceeded.
Shared Responsibilities, Not an “Extreme Case”
None of these defenses rests on a single actor. The safety of an autonomous system is distributed among sensor manufacturers, perception-model developers, integrators who assemble the vehicle or robot, operators who deploy it, and regulators who set the framework. Each bears different obligations: document limits, test adversarial scenarios, keep uncertainty records, and demonstrate how the system responds when its sensors disagree.
In the United States, the guide Automated Driving Systems 2.0: A Vision for Safety from the NHTSA, published in 2021, recommends validating the entire system within its intended operating domain (the so-called Operational Design Domain), including normal driving behaviors, collision avoidance, minimum-risk operations, and subsystem failures. That validation may combine simulation, test tracks, on-road trials, and potentially independent third-party testing.
Two nuances are worth highlighting. The first is that this guidance is voluntary: it does not amount to a federal pre-certification guaranteeing the safety of every system, though the NHTSA retains authority over defects, recalls, and regulatory compliance. The second stems directly from the study: if heat surfaces, reflections, or industrial heat sources can appear in the everyday operating domain of a drone, delivery robot, or vehicle, labeling these scenarios as “extreme cases” would be insufficient. A foreseeable thermal scene should be part of standard testing, not a list of unlikely exceptions.
The Core Lesson
We have grown accustomed to evaluating artificial perception safety in terms of accuracy: how many pedestrians a system detects, with what precision it classifies an obstacle. This work shifts the focus to something harder to quantify and more decisive. The striking finding is not that a camera can misfire—an expectation we already hold—but that the reliability of an autonomous architecture will increasingly depend on its capacity to recognize, in time, when one of its sensors has ceased to be a trustworthy source of knowledge.
A truly robust system is not the one that never errs, but the one that knows how to doubt itself when the world stops aligning with what its instruments report. Teaching machines to distrust their own gaze could be, paradoxically, one of the greatest safety breakthroughs awaiting us.
References
Bhupathiraju, Sri Hrushikesh Varma; Xie, Shaoyuan; Clifford, Michael; Chen, Qi Alfred; Sugawara, Takeshi; and Rampazzi, Sara. “The Heat is On: Understanding and Mitigating Vulnerabilities of Thermal Image Perception in Autonomous Systems.” NDSS Symposium 2026. DOI: 10.14722/ndss.2026.230330. Available at: https://www.ndss-symposium.org/wp-content/uploads/2026-s330-paper.pdf
University of Florida. “Thermal cameras used in drones and robots can be tricked by heat sources, study finds.” Institutional press release, March 16, 2026. Available at: https://news.ufl.edu/2026/03/thermal-cameras/
National Highway Traffic Safety Administration. “Automated Driving Systems 2.0: A Vision for Safety.” U.S. Department of Transportation, 2021. Available at: https://www.nhtsa.gov/document/automated-driving-systems-20-voluntary-guidance
National Highway Traffic Safety Administration. “Automated Driving Systems.” U.S. Department of Transportation, institutional webpage accessed in 2026. Available at: https://www.nhtsa.gov/vehicle-manufacturers/automated-driving-systems
