before you move on, please check, that this first example works. link a mediafile with video and audio. doublecheck your uri!
copy the sourcecode into a .sh (or .bat) file.
SRCA="file:///c:/msys64/home/01_clock.mp4"
gst-launch-1.0 \
uridecodebin uri=$SRCA name=media \
\
media. ! queue ! videoconvert ! queue ! autovideosink \
media. ! queue ! audioconvert ! audioresample ! queue ! autoaudiosink
the code above works in linux as the variable and linebreaks are written that way, in windows you have to use set, %% and ^.
set SRCA="file:///c:/msys64/home/01_clock.mp4"
gst-launch-1.0 uridecodebin uri=%SRCA% name=media^
media. ! queue ! videoconvert ! queue ! autovideosink^
media. ! queue ! audioconvert ! audioresample ! queue ! autoaudiosink
a compositor example with a background image, 2 videostreams and a logo. no audio.
SRCA="file:///c:/msys64/home/01_clock.mp4"
SRCB="file:///c:/msys64/home/02_lakme.mkv"
SRCBG="file:///c:/msys64/home/bg_blue.png"
SRCLogo="file:///c:/msys64/home/logo.png"
GST_DEBUG=2 gst-launch-1.0 compositor name=comp \
sink_0::alpha=1.0 sink_0::xpos=0 sink_0::ypos=0 sink_0::width=960 sink_0::height=540 \
sink_1::alpha=0.5 sink_1::xpos=0 sink_1::ypos=0 sink_1::width=960 sink_1::height=540 \
sink_2::alpha=1.0 sink_2::xpos=600 sink_2::ypos=320 sink_2::width=320 sink_2::height=180 \
sink_3::alpha=1.0 sink_3::xpos=890 sink_3::ypos=10 sink_3::width=60 sink_3::height=60 \
! videoscale ! video/x-raw,width=960,height=540 ! videoconvert ! autovideosink \
uridecodebin uri=$SRCBG ! queue ! videoscale ! video/x-raw,width=960,height=540 ! imagefreeze ! comp.sink_0 \
uridecodebin uri=$SRCA ! queue ! videoscale ! video/x-raw,width=960,height=540 ! videoconvert ! queue ! comp.sink_1 \
uridecodebin uri=$SRCB ! queue ! videoscale ! video/x-raw,width=320,height=180 ! videoconvert ! queue ! comp.sink_2 \
uridecodebin uri=$SRCLogo ! queue ! videoscale ! video/x-raw,width=60,height=60 ! imagefreeze ! comp.sink_3 \
if errors occur, check the file-uri, change the GST_DEBUG to 3 and look for file not found or could not open. look for media-resolutions. these examples where built and tested under windows 11 mingw64.
this is the same compositor example but filled with named pad-links and, for readabilty, empty lines
GST_DEBUG=2 \
gst-launch-1.0 \
compositor name=comp \
sink_0::alpha=1.0 sink_0::xpos=0 sink_0::ypos=0 sink_0::width=960 sink_0::height=540 \
sink_1::alpha=0.5 sink_1::xpos=0 sink_1::ypos=0 sink_1::width=960 sink_1::height=540 \
sink_2::alpha=1.0 sink_2::xpos=600 sink_2::ypos=320 sink_2::width=320 sink_2::height=180 \
sink_3::alpha=1.0 sink_3::xpos=890 sink_3::ypos=10 sink_3::width=60 sink_3::height=60 \
! videoscale ! video/x-raw,width=960,height=540 ! videoconvert ! mux_out. \
\
uridecodebin uri=$SRCA name=videoA \
uridecodebin uri=$SRCB name=videoB \
\
uridecodebin uri=$SRCBG ! queue ! videoscale ! video/x-raw,width=960,height=540 ! imagefreeze ! comp.sink_0 \
videoA. ! queue ! videoscale ! video/x-raw,width=960,height=540 ! videoconvert ! queue ! comp.sink_1 \
videoB. ! queue ! videoscale ! video/x-raw,width=320,height=180 ! videoconvert ! queue ! comp.sink_2 \
uridecodebin uri=$SRCLogo ! queue ! videoscale ! video/x-raw,width=60,height=60 ! imagefreeze ! comp.sink_3 \
\
autovideosink name=mux_outwe re adding audio of one of the videos
SRCA="file:///c:/msys64/home/01_clock.mp4"
SRCB="file:///c:/msys64/home/02_lakme.mkv"
SRCBG="file:///c:/msys64/home/bg_blue.png"
SRCLogo="file:///c:/msys64/home/logo.png"
GST_DEBUG=2 \
gst-launch-1.0 \
compositor name=comp \
sink_0::alpha=1.0 sink_0::xpos=0 sink_0::ypos=0 sink_0::width=960 sink_0::height=540 \
sink_1::alpha=0.5 sink_1::xpos=0 sink_1::ypos=0 sink_1::width=960 sink_1::height=540 \
sink_2::alpha=1.0 sink_2::xpos=600 sink_2::ypos=320 sink_2::width=320 sink_2::height=180 \
sink_3::alpha=1.0 sink_3::xpos=890 sink_3::ypos=10 sink_3::width=60 sink_3::height=60 \
! videoscale ! video/x-raw,width=960,height=540 ! videoconvert ! mux_out. \
\
uridecodebin uri=$SRCA name=videoA \
uridecodebin uri=$SRCB name=videoB \
\
uridecodebin uri=$SRCBG ! queue ! videoscale ! video/x-raw,width=960,height=540 ! imagefreeze ! comp.sink_0 \
videoA. ! queue ! videoscale ! video/x-raw,width=960,height=540 ! videoconvert ! queue ! comp.sink_1 \
videoB. ! queue ! videoscale ! video/x-raw,width=320,height=180 ! videoconvert ! queue ! comp.sink_2 \
uridecodebin uri=$SRCLogo ! queue ! videoscale ! video/x-raw,width=60,height=60 ! imagefreeze ! comp.sink_3 \
\
videoB. ! queue ! audioconvert ! audioresample ! queue ! autoaudiosink \
\
autovideosink name=mux_out
time to test saving muxed audio and video into a file. stop the encoding with CTRL+C, ensuring the file will be finalized accordingly. look out if the nv264enc encoder works for you
SRCA="file:///c:/msys64/home/01_clock.mp4"
SRCB="file:///c:/msys64/home/02_lakme.mkv"
SRCBG="file:///c:/msys64/home/bg_blue.png"
SRCLogo="file:///c:/msys64/home/logo.png"
SRCSave="example.mkv"
GST_DEBUG=2 \
gst-launch-1.0 -e \
compositor name=comp \
sink_0::alpha=1.0 sink_0::xpos=0 sink_0::ypos=0 sink_0::width=960 sink_0::height=540 \
sink_1::alpha=0.5 sink_1::xpos=0 sink_1::ypos=0 sink_1::width=960 sink_1::height=540 \
sink_2::alpha=1.0 sink_2::xpos=600 sink_2::ypos=320 sink_2::width=320 sink_2::height=180 \
sink_3::alpha=1.0 sink_3::xpos=890 sink_3::ypos=10 sink_3::width=60 sink_3::height=60 \
! videoscale ! video/x-raw,width=960,height=540,framerate=25/1 ! videoconvert ! enc_video. \
\
uridecodebin uri=$SRCA name=videoA \
uridecodebin uri=$SRCB name=videoB \
\
uridecodebin uri=$SRCBG ! queue ! videoscale ! video/x-raw,width=960,height=540 ! imagefreeze ! comp.sink_0 \
videoA. ! queue ! videoconvert ! videoscale ! video/x-raw,width=960,height=540 ! videoconvert ! queue ! comp.sink_1 \
videoB. ! queue ! videoscale ! video/x-raw,width=320,height=180 ! videoconvert ! queue ! comp.sink_2 \
uridecodebin uri=$SRCLogo ! queue ! videoscale ! video/x-raw,width=60,height=60 ! imagefreeze ! comp.sink_3 \
\
videoB. ! queue ! audioconvert ! audioresample ! faac ! queue ! mux_out. \
nvh264enc name=enc_video bitrate=1200 ! h264parse ! queue ! mux_out. \
\
matroskamux name=mux_out ! filesink location=$SRCSave
finally, a rtsp stream with video and audio.
in this example sending the stream to mediamtx as a stream-server-distributor (you have to install rtspclientserver as its not in the standard packages and of course mediamtx). for webrtc streams you need to change the faac encoder to opusenc
SRCA="file:///c:/msys64/home/01_clock.mp4"
SRCB="file:///c:/msys64/home/02_lakme.mkv"
SRCBG="file:///c:/msys64/home/bg_blue.png"
SRCLogo="file:///c:/msys64/home/logo.png"
RTSPURL="rtsp://192.168.2.6:8554/mystream"
GST_DEBUG=2 \
gst-launch-1.0 -e \
compositor name=comp \
sink_0::alpha=1.0 sink_0::xpos=0 sink_0::ypos=0 sink_0::width=960 sink_0::height=540 \
sink_1::alpha=0.5 sink_1::xpos=0 sink_1::ypos=0 sink_1::width=960 sink_1::height=540 \
sink_2::alpha=1.0 sink_2::xpos=600 sink_2::ypos=320 sink_2::width=320 sink_2::height=180 \
sink_3::alpha=1.0 sink_3::xpos=890 sink_3::ypos=10 sink_3::width=60 sink_3::height=60 \
! videoscale ! video/x-raw,width=960,height=540,framerate=25/1 ! videoconvert ! enc_video. \
\
uridecodebin uri=$SRCA name=videoA \
uridecodebin uri=$SRCB name=videoB \
\
uridecodebin uri=$SRCBG ! queue ! videoscale ! video/x-raw,width=960,height=540 ! imagefreeze ! comp.sink_0 \
videoA. ! queue ! videoconvert ! videoscale ! video/x-raw,width=960,height=540 ! videoconvert ! queue ! comp.sink_1 \
videoB. ! queue ! videoscale ! video/x-raw,width=320,height=180 ! videoconvert ! queue ! comp.sink_2 \
uridecodebin uri=$SRCLogo ! queue ! videoscale ! video/x-raw,width=60,height=60 ! imagefreeze ! comp.sink_3 \
\
videoB. ! queue ! audioconvert ! audioresample ! faac ! queue ! stream.sink_1 \
nvh264enc name=enc_video bitrate=1200 ! queue ! stream.sink_0 \
\
rtspclientsink name=stream location=$RTSPURL