Streaming Protocol and Codec Decision Matrix
Relevant data
Bandwidth requirements of various video streaming protocols at 1 FPS (e.g. RTSP, RTMP, WebRTC, etc.), including video codecs (AV1, etc.).
Include the option where we just send images individually (i.e. through a raw socket), which includes research on various image codecs (e.g. JPEG, PNG, JPEG XL, AVIF, etc.).
Options Considered
There are some other methods such as MPEG-DASHS that were previously on this table that are no longer considered due to extremely high latency or lack of demonstratable functionality to be found.
| Real-Time Messaging Protocol (RTMP) | Real-Time Streaming Protocol (RTSP) | WebRTC | Secure Reliable Transport (SRT) | TCP / UDP | HTTP | Sending images over socket |
---|---|---|---|---|---|---|---|
Classification | Legacy Protocol | Legacy Protocol | Modern Streaming Protocol | Modern Streaming Protocol |
|
|
|
Description | Works on top of TCP, maintains persistent connections and allows low-latency communication. | Main task is not streaming, rather it is a control protocol that works in conjunction with RTP and RTCP, which often also works on TCP. | Supported by most modern applications. Primarily works over UDP, TCP may be supported | Uses the UDP transmission protocol. However, has extra layers of optimization and security | Same category because they are treated very similarly | Streaming over HTTP | Self-explanatory |
Video Codecs | H.264, VP8 | H.265, H.264, VP9, VP8 | H.264, VP8, VP9 | Codec-agnostic | H.264 and H.265 | H.264 and H.265 |
|
Latency | 5 - 30s (LL-RTMP can be around 2-3s) | 2-5s | < 0.5s | < 3s | few hundred ms | few seconds, but probably heavily depends on context since the example below has it working | Higher latency and less continuity than using a streaming protocol |
Implementation & Difficulty | Out of date for most platforms, latency could be a problem. Seems RTSP is favoured in most cases | Example: Unlimited Range Video (FPV) Link for Your Drone using Raspberry Pi Camera
Probably more structurally complex than TCP/UDP/HTTP | Example: Drone control by WebRTC on web application | Theoretically would be good. However, SRT is not standardized and no demonstrated implementation is found | Example: Unlimited Range Video (FPV) Link for Your Drone using Raspberry Pi Camera | Example: Unlimited Range Video (FPV) Link for Your Drone using Raspberry Pi Camera |
|
Prevalence | Not used very often | Seems like a good choice, there are a lot of sources using RTSP | Some sources on it, not as much | Next to none | Quite a common approach | A few sources | There is 0 information on this |