GStreamer:RTP RTSP
lastmod 2023-01-20created 2023-01-202 edits
GStreamer has excellent support for both RTP and RTSP, and its RTP/RTSP stack has proven itself over the years in wide production use in a variety of mission-critical and low-latency scenarios, from small embedded devices to large-scale video conferencing.
GStreamer RTSP server
GStreamer's RTSP server (gst-rtsp-server) is a feature-rich and easy-to-use library that allows applications to implement a complete RTSP server with just a few lines of code.
It is multi-threaded, scalable, and flexible, and provides support for static or dynamic mount points, authentication, retransmission (rtx), encryption (srtp, secure RTP), UDP unicast and multicast, TCP interleaving, seeking, and optionally cgroup integration for advanced resource management and control. It can also distribute a GStreamer net client clock to GStreamer RTSP clients to facilitate multi-device synchronization.
GStreamer RTSP client
The GStreamer rtspsrc element in gst-plugins-good is GStreamer's high-level RTSP client abstraction. It can be used directly as a standalone element, or through playbin by passing an rtsp:// URI to playbin. rtspsrc provides a number of GObject properties that can be configured in all kinds of different ways. In particular, the "latency" property, which configures the default jitterbuffer latency, can be set to a lower value to achieve low latency.
RTP components
Most of GStreamer's main RTP components are in gst-plugins-good.
- The rtpmanager plugin contains elements such as rtpbin and rtpjitterbuffer.
- The rtp plugin contains RTP payloading and depayloading elements for various codecs and container formats.
Some lower-level libraries in gst-plugins-base:
- The GStreamer RTP library contains the RTP payloader/depayloader base classes and functions for handling RTP and RTCP buffers.
- The GStreamer MIKEY library contains helper functions for handling MIKEY messages for secure RTP.
- The GStreamer RTSP library contains the low-level RTSP functionality used by gst-rtsp-server and by high-level objects such as rtspsrc.
- The GStreamer SDP library contains utility functions for parsing and generating SDP messages.
Some of the main components are:
- rtpbin is a high-level RTP component that supports sending and receiving, or only sending or receiving data, with or without RTCP support. It is a bin that does all the work. It adapts dynamically to the requirements based on the requested pads. It also contains an rtpjitterbuffer.
- rtpjitterbuffer is an RTP buffer that controls network jitter and reorders packets. It also dumps packets that arrive too late, handles packet retransmission and lost-packet notifications, and adjusts for sender-receiver clock drift.
- rtpptdemux is an element usually placed on an rtpbin src pad that detects new payload types arriving in the RTP stream. It then creates a pad for the new payload, and a depayloader/decoder pipeline can be connected to that pad.
- rtpssrcdemux is an element usually placed on an rtpbin src pad that detects new SSRCs arriving in the RTP stream. It then creates a pad for the new SSRC, and a depayloader/decoder pipeline can be connected to that pad.
- GstRTPBaseDepayload is the base class for RTP depayloaders.
- GstRTPBasePayload is the base class for RTP payloaders.
- GstRTPBaseAudioPayload is the base class for audio RTP payloaders.
Many RTP elements assume they receive RTP buffers with GstNetAddressMeta metadata set (since udpsrc creates it).
Good Plugins
rtp from GStreamer Good Plug-ins
- asteriskh263 - Extracts H263 video from RTP and encodes in Asterisk H263 format
- rtpac3depay - Extracts AC3 audio from RTP packets (RFC 4184)
- rtpac3pay - Payload AC3 audio as RTP packets (RFC 4184)
- rtpamrdepay - Extracts AMR or AMR-WB audio from RTP packets (RFC 3267)
- rtpamrpay - Payload-encode AMR or AMR-WB audio into RTP packets (RFC 3267)
- rtpbvdepay - Extracts BroadcomVoice audio from RTP packets (RFC 4298)
- rtpbvpay - Packetize BroadcomVoice audio streams into RTP packets (RFC 4298)
- rtpceltdepay - Extracts CELT audio from RTP packets
- rtpceltpay - Payload-encodes CELT audio into a RTP packet
- rtpdvdepay - Depayloads DV from RTP packets (RFC 3189)
- rtpdvpay - Payloads DV into RTP packets (RFC 3189)
- rtpg722depay - Extracts G722 audio from RTP packets
- rtpg722pay - Payload-encode Raw audio into RTP packets (RFC 3551)
- rtpg723depay - Extracts G.723 audio from RTP packets (RFC 3551)
- rtpg723pay - Packetize G.723 audio into RTP packets
- rtpg726depay - Extracts G.726 audio from RTP packets
- rtpg726pay - Payload-encodes G.726 audio into a RTP packet
- rtpg729depay - Extracts G.729 audio from RTP packets (RFC 3551)
- rtpg729pay - Packetize G.729 audio into RTP packets
- rtpgsmdepay - Extracts GSM audio from RTP packets
- rtpgsmpay - Payload-encodes GSM audio into a RTP packet
- rtpgstdepay - Extracts GStreamer buffers from RTP packets
- rtpgstpay - Payload GStreamer buffers as RTP packets
- rtph261depay - Extracts H261 video from RTP packets (RFC 4587)
- rtph261pay - Payload-encodes H261 video in RTP packets (RFC 4587)
- rtph263depay - Extracts H263 video from RTP packets (RFC 2190)
- rtph263pay - Payload-encodes H263 video in RTP packets (RFC 2190)
- rtph263pdepay - Extracts H263/+/++ video from RTP packets (RFC 4629)
- rtph263ppay - Payload-encodes H263/+/++ video in RTP packets (RFC 4629)
- rtph264depay - Extracts H264 video from RTP packets (RFC 3984)
- rtph264pay - Payload-encode H264 video into RTP packets (RFC 3984)
- rtph265depay - Extracts H265 video from RTP packets (RFC 7798)
- rtph265pay - Payload-encode H265 video into RTP packets (RFC 7798)
- rtphdrextcolorspace - Extends RTP packets with color space and high dynamic range (HDR) information.
- rtpilbcdepay - Extracts iLBC audio from RTP packets (RFC 3952)
- rtpilbcpay - Packetize iLBC audio streams into RTP packets
- rtpisacdepay - Extracts iSAC audio from RTP packets
- rtpisacpay - Payload-encodes iSAC audio into a RTP packet
- rtpj2kdepay - Extracts JPEG 2000 video from RTP packets (RFC 5371)
- rtpj2kpay - Payload-encodes JPEG 2000 pictures into RTP packets (RFC 5371)
- rtpjpegdepay - Extracts JPEG video from RTP packets (RFC 2435)
- rtpjpegpay - Payload-encodes JPEG pictures into RTP packets (RFC 2435)
- rtpklvdepay - Extracts KLV (SMPTE ST 336) metadata from RTP packets
- rtpklvpay - Payloads KLV (SMPTE ST 336) metadata as RTP packets
- rtpL16depay - Extracts raw audio from RTP packets
- rtpL16pay - Payload-encode Raw audio into RTP packets (RFC 3551)
- rtpL24depay - Extracts raw 24-bit audio from RTP packets
- rtpL24pay - Payload-encode Raw 24-bit audio into RTP packets (RFC 3190)
- rtpL8depay - Extracts raw audio from RTP packets
- rtpL8pay - Payload-encode Raw audio into RTP packets (RFC 3551)
- rtpldacpay - Payload LDAC audio as RTP packets
- rtpmp1sdepay - Extracts MPEG1 System Streams from RTP packets (RFC 3555)
- rtpmp2tdepay - Extracts MPEG2 TS from RTP packets (RFC 2250)
- rtpmp2tpay - Payload-encodes MPEG2 TS into RTP packets (RFC 2250)
- rtpmp4adepay - Extracts MPEG4 audio from RTP packets (RFC 3016)
- rtpmp4apay - Payload MPEG4 audio as RTP packets (RFC 3016)
- rtpmp4gdepay - Extracts MPEG4 elementary streams from RTP packets (RFC 3640)
- rtpmp4gpay - Payload MPEG4 elementary streams as RTP packets (RFC 3640)
- rtpmp4vdepay - Extracts MPEG4 video from RTP packets (RFC 3016)
- rtpmp4vpay - Payload MPEG-4 video as RTP packets (RFC 3016)
- rtpmpadepay - Extracts MPEG audio from RTP packets (RFC 2038)
- rtpmpapay - Payload MPEG audio as RTP packets (RFC 2038)
- rtpmparobustdepay - Extracts MPEG audio from RTP packets (RFC 5219)
- rtpmpvdepay - Extracts MPEG video from RTP packets (RFC 2250)
- rtpmpvpay - Payload-encodes MPEG2 ES into RTP packets (RFC 2250)
- rtpopusdepay - Extracts Opus audio from RTP packets
- rtpopuspay - Puts Opus audio in RTP packets
- rtppcmadepay - Extracts PCMA audio from RTP packets
- rtppcmapay - Payload-encodes PCMA audio into a RTP packet
- rtppcmudepay - Extracts PCMU audio from RTP packets
- rtppcmupay - Payload-encodes PCMU audio into a RTP packet
- rtpqcelpdepay - Extracts QCELP (PureVoice) audio from RTP packets (RFC 2658)
- rtpqdm2depay - Extracts QDM2 audio from RTP packets (no RFC)
- rtpreddec - Decode Redundant Audio Data (RED)
- rtpredenc - Encode Redundant Audio Data (RED)
- rtpsbcdepay - Extracts SBC audio from RTP packets
- rtpsbcpay - Payload SBC audio as RTP packets
- rtpsirendepay - Extracts Siren audio from RTP packets
- rtpsirenpay - Packetize Siren audio streams into RTP packets
- rtpspeexdepay - Extracts Speex audio from RTP packets
- rtpspeexpay - Payload-encodes Speex audio into a RTP packet
- rtpstorage - Helper element for various purposes (ex. recovering from packet loss using RED/FEC). Saves given number of RTP packets. Should be instantiated before jitterbuffer
- rtpstreamdepay - Depayloads RTP/RTCP packets for streaming protocols according to RFC4571
- rtpstreampay - Payloads RTP/RTCP packets for streaming protocols according to RFC4571
- rtpsv3vdepay - Extracts SVQ3 video from RTP packets (no RFC)
- rtptheoradepay - Extracts Theora video from RTP packets (draft-01 of RFC XXXX)
- rtptheorapay - Payload-encode Theora video into RTP packets (draft-01 RFC XXXX)
- rtpulpfecdec - Decodes RTP FEC (RFC5109)
- rtpulpfecenc - Encodes RTP FEC (RFC5109)
- rtpvorbisdepay - Extracts Vorbis Audio from RTP packets (RFC 5215)
- rtpvorbispay - Payload-encode Vorbis audio into RTP packets (RFC 5215)
- rtpvp8depay - Extracts VP8 video from RTP packets)
- rtpvp8pay - Puts VP8 video in RTP packets
- rtpvp9depay - Extracts VP9 video from RTP packets)
- rtpvp9pay - Puts VP9 video in RTP packets)
- rtpvrawdepay - Extracts raw video from RTP packets (RFC 4175)
- rtpvrawpay - Payload raw video as RTP packets (RFC 4175)