Video Encoding and Streaming Architecture: H.264, H.265, RTSP and SEI

The video pipeline is where a payload’s image quality is either preserved or quietly destroyed. Codec, bitrate, GOP structure and latency settings interact, and the defaults that look best on a bench — high bitrate, long GOP — are often the worst choices over a real radio link.

Key takeaways

  • H.265 buys roughly half the bitrate at equal quality, and costs decode compatibility and latency on older ground equipment.
  • Bitrate is meaningful only against resolution and scene motion; a fixed number is not a quality specification.
  • GOP structure is a latency decision, not a quality one — long GOPs compress better and delay recovery after packet loss.
  • Telemetry carried inside the video stream keeps the metadata aligned with the frame it describes, which matters as soon as anyone geolocates a finding.

H.264 or H.265

H.265 delivers roughly the same perceived quality at 40–50 % of the bitrate, which on a bandwidth-limited downlink is decisive. The costs are encoder latency, decoder compatibility and power. Some ground software, older recorders and many third-party VMS platforms still handle H.264 more reliably.

The practical rule: use H.265 when the link is the constraint and you control both ends; use H.264 when interoperability matters more than bandwidth. Payloads that support both per-stream let you run H.265 to the operator and H.264 to the recorder.

Bitrate and resolution, in context

Resolution / rateH.264 bitrateH.265 bitrateTypical useLatency band
1920×1080 @306–10 Mbps3–5 MbpsPrimary operator view80–180 ms
1920×1080 @255–8 Mbps2.5–4 MbpsRecording / VMS80–180 ms
1280×720 @302.5–4 Mbps1.2–2 MbpsDegraded-link fallback60–140 ms
3840×2160 @3025–40 Mbps12–20 MbpsInspection capture150–350 ms
640×512 thermal @301.5–3 Mbps0.8–1.5 MbpsThermal channel60–140 ms
PiP composite 1080p7–11 Mbps3.5–5.5 MbpsDual-channel view100–200 ms
End-to-end latency includes sensor readout, encode, transport and decode. Encoder alone is typically 20–60 ms.

GOP structure is a latency decision

A long group of pictures — many predicted frames between keyframes — is bandwidth-efficient and fragile. Lose a keyframe over a marginal link and the picture stays broken until the next one arrives, which on a 2-second GOP means 2 seconds of visible corruption.

A short GOP costs bitrate but recovers quickly. For live operational video over a radio link, 0.5–1 second GOP is a reasonable compromise; for recorded inspection footage over a stable connection, longer is fine.

Zero-latency or low-delay encoder modes disable B-frames and frame reordering. They cost perhaps 10–15 % bitrate efficiency and remove 30–80 ms of pipeline delay. For any closed-loop task — manual tracking, precision pointing, FPV — take the trade. The full latency chain is broken down in video latency in UAV payload systems.

Carrying telemetry inside the video

SEI (supplemental enhancement information) is a standard container inside H.264 and H.265 streams for arbitrary per-frame metadata. A payload can stamp each frame with gimbal angles, target coordinate, zoom position, laser range and timestamp — and that data travels with the frame through any standards-compliant pipeline.

This solves a problem that otherwise requires a separate synchronised telemetry channel and a fragile alignment step downstream. If the metadata rides in the frame, it cannot drift out of sync with the frame.

The practical caution: many players and recorders discard SEI silently on transcode. Verify end to end before designing a workflow that depends on it.

How this shows up in our payloads

Our payloads output H.264 and H.265 over RTSP with per-stream configuration, so the operator view and the recording stream can use different codecs, bitrates and GOP settings. Models such as OP-80U carry shooting-point coordinate in EXIF for stills and SEI for video, so downstream tools inherit position without a separate telemetry merge. Micro pods such as MV-2P expose the same interfaces within a smaller power budget.

Technology: multi-sensor time synchronization and the thermal image processing pipeline.

Field practice: ONVIF and IP video on drone payloads, latency budgets by mission and thermal drone data management.

Configuring the stream for the mission you are flying

There is no single correct encoder configuration, and the default shipped by most payloads is tuned for a bench demonstration: high bitrate, long GOP, quality-optimised. That configuration produces beautiful video on a cable and poor video over a marginal radio link.

Match the settings to the constraint. When the link is the bottleneck, drop resolution before you drop bitrate — a clean 720p stream is far more usable than a blocky, artefact-laden 1080p one at the same bitrate. When latency is the constraint, disable B-frames and use a low-delay encoder mode even at a bitrate cost. When the deliverable is an inspection record, use the highest quality the storage allows and stop worrying about latency entirely, because nobody is closing a control loop on it.

Run separate profiles for separate purposes where the payload allows it. The operator view and the archival recording have almost opposite requirements, and forcing one configuration to serve both compromises both.

  • Link-limited: reduce resolution first, shorten GOP to 0.5–1 s, and prefer H.265 if both ends support it.
  • Latency-critical: low-delay mode, no B-frames, short GOP; accept the bitrate penalty.
  • Inspection capture: maximum quality, long GOP acceptable, record locally rather than relying on the downlink.
  • Mixed use: run two profiles — operator stream and recording stream — with different codecs and bitrates.
  • Verify end to end that your recorder preserves the metadata you depend on before designing a workflow around it.

FAQ

Should I use H.265 or H.264?

H.265 if downlink bandwidth is your constraint and you control both ends — it gives comparable quality at roughly half the bitrate. H.264 if you need broad compatibility with existing recorders and VMS platforms. Payloads that support both per stream let you have each where it fits.

A long GOP. Losing a keyframe leaves the decoder predicting from corrupted data until the next one arrives. Shortening the GOP to 0.5–1 second costs some bitrate efficiency and dramatically improves recovery over marginal links.

Can telemetry travel inside the video stream?

Yes, via SEI in H.264 and H.265. Gimbal angles, target coordinates and timestamps can be stamped per frame so metadata cannot drift out of sync with imagery. Confirm your recorder and player preserve SEI, because many transcoding steps silently discard it.