AI Target Tracking on UAV Payloads: How It Actually Works

AI tracking on a modern payload does two jobs: it finds targets — human and vehicle detection painted as boxes in the feed — and it holds them, steering the gimbal automatically while the aircraft moves. Together they turn a two-operator task into a one-operator task.

• 16×16 pxMinimum trackable object
• <40 msObject identification delay
• ±32 pxTracking speed per field
• OnboardWhere inference runs

Detection: The Automatic Spotter

Onboard neural networks classify humans and vehicles in real time, painting boxes on the feed as they go. The value is not that the network sees better than a person — it does not — but that it never stops looking. A human operator scanning a wide scene at altitude has an attention budget that degrades over a long mission, and the targets that get missed are exactly the small, slow, unremarkable ones.

This is the backbone of patrol automation on payloads like the OP-80A and OP-90A. In a perimeter or corridor pattern the aircraft flies a predictable route while the detector watches everything in frame, and the operator responds to flags rather than staring continuously.

Running inference on the payload rather than in a ground application matters more than it sounds. Onboard detection sees full-resolution frames before compression and transmission, and it keeps working when the link degrades. Onboard AI inference architecture covers the compute, latency and power triangle that decides what fits in a pod.

Tracking: The Automatic Cameraman

Once locked — usually by clicking the target in the feed — the gimbal keeps it centred through aircraft repositioning, target motion and brief occlusions, with reacquisition logic for when the subject reappears. The operator stops flying the camera and starts thinking about the incident, which is the actual productivity gain.

The published figures show what the tracker is working with across the line.

Payload	Object size	Identification delay	Tracking speed
MV-2M	16×16 to 128×128 px	<40 ms	±32 px / field
MV-2P	16×16 to 128×128 px	<40 ms	±32 px / field
OP-80A	16×16 to 128×128 px	<40 ms	±32 px / field
OP-80U	16×16 to 128×128 px	<40 ms	±32 px / field
OP-90A	16×16 to 128×128 px	<40 ms	±32 px / field
LX-6U	16×16 to 128×128 px	<40 ms	±32 px / field
OP-125A	—	—	>24 px / frame
LX-9B	—	—	>24 px / frame

Two numbers deserve attention. The 16×16 pixel minimum means tracking range is a function of lens and sensor resolution, not of the tracker: a target below that size in frame cannot be locked no matter how obvious it looks to you. The 128×128 maximum is the other end — a subject filling much of the frame at close range can also fall outside the window, which is why locks sometimes drop as you close in rather than as you pull away.

The <40 ms identification delay is what makes the loop feel immediate. Anything much slower and the gimbal is chasing where the target was, which at high zoom is visible as lag and overshoot.

Tracking on Thermal as Well as Visible

On multi-sensor payloads like the OP-125A and LX-9B, detection and tracking run on visible and thermal feeds alike. This is more consequential than a feature-list line suggests, because the two channels fail under opposite conditions.

Thermal tracking excels at night when visible contrast fails: a warm subject against cool ground is a high-contrast target in LWIR and nearly nothing in a starlight image. Conversely, a subject on sun-heated asphalt in the afternoon may be thermally invisible while remaining perfectly obvious in the visible channel. A tracker that can run on either lets the operator pick whichever channel currently has the signal.

For the payload to switch cleanly, the two channels must be aligned — a box drawn on the thermal feed has to land on the same object in the visible one. That is what boresight alignment exists to guarantee, and it is why dual-channel display modes and tracking are closely related capabilities.

Honest Limits

Dense crowds, target crossings, long occlusions and extreme scale changes still break locks. Each has a characteristic signature once you have seen it a few times.

Dense crowds give the tracker many similar candidates and it will eventually pick the wrong one — usually at the moment two people pass. Crossings are the classic failure: two subjects intersect, and the lock leaves with whichever one the appearance model preferred. Long occlusions exhaust the reacquisition window; brief ones behind a tree are fine, thirty seconds behind a building generally are not. Extreme scale changes — a rapid descent, or a subject running directly away — move the target outside the size window that the detector operates in.

Good operators treat AI as a workload reducer, not an autonomous system. The practical discipline is to keep watching the tracked feed rather than trusting it, and to re-establish the lock deliberately after any event that might have swapped the subject. In a law-enforcement context especially, a track that silently transferred to the wrong person is worse than no track at all — see thermal drones in law enforcement.

What Tracking Does Not Give You

A lock is not a coordinate. Tracks overlay the recorded stream, but converting “the camera is pointed at this object” into “the object is at these coordinates” requires pairing the gimbal angles with aircraft telemetry and, for real precision, a rangefinder measurement — as the LRF guide and the geolocation error budget set out.

Nor is a detection an identification. A box labelled “person” means the network classified a shape as human at some confidence; it does not mean the person is your subject. The DRI framework applies unchanged — the pixel count that supports detection is roughly a quarter of what supports identification, and no amount of inference changes that arithmetic.

And it does not remove the need for a competent operator. What it removes is the second operator: the one who used to fly the camera while the first thought about the incident. That is a real and substantial gain, and it is worth describing accurately rather than as autonomy.

Related reading

• Onboard AI Inference Architecture: The Compute, Latency and Power Triangle
• EO/IR Sensor Fusion and Boresight Alignment
• Target Geolocation Error Budget
• DRI Ranges Explained
• Thermal Drones in Law Enforcement
• PiP, Fusion and Split View: Dual-Channel Display Modes
• Why a Laser Rangefinder Belongs on Your UAV Payload
• Video Latency in UAV Payload Systems
• OP-90A — multi-sensor pod with AI tracking
• OP-125A — 30x zoom with AI tracking
• LX-9B — multi-sensor pod with AI tracking
• Border and Perimeter Surveillance Drones

FAQ

Does AI tracking work on the thermal channel?

Yes. On multi-sensor payloads like the OP-125A and LX-9B, detection and tracking run on visible and thermal feeds alike, and thermal tracking excels at night when visible contrast fails. The two channels fail under opposite conditions — a subject invisible on sun-heated asphalt in thermal may be obvious in visible, and vice versa after dark — so being able to track on either is a genuine operational advantage rather than a specification line.

Is tracked data recorded?

Tracks overlay the recorded stream, so the box and its history are visible in playback. Coordinate output is a separate matter: it depends on pairing the gimbal angles with aircraft telemetry and, for useful precision, a laser rangefinder measurement. A track alone tells you where the camera was pointed, not where the object was — those are different claims and only the second one belongs in a report.

Why does the lock sometimes drop as I get closer?

Because the tracker operates on a defined object size window — the published range is 16×16 to 128×128 pixels. A subject that fills much of the frame at close range exceeds the upper bound just as a distant one falls below the lower bound. Rapid scale change during a descent is the common trigger. Re-establishing the lock at the new range is the normal remedy, and it is worth expecting rather than treating as a fault.

Questions about the technology? Talk to our engineers — we reply within 2 business days.