I am trying to graduate beyond the ball example but the pipeline seems stuck and does not start. What am I doing wrong?

./whip-client -u http://serverexample.com:7080/whip/endpoint/drone -t veryverysecret -V "multifilesrc location=drone.mp4 loop=true ! qtdemux ! h264parse ! rtph264pay ! application/x-rtp,media=video,encoding-name=H264,payload=96" -S stun://stun.l.google.com:19302

--------------------
Simple WHIP client
------------------

WHIP endpoint:  http://serverexample.com:7080/whip/endpoint/drone
Bearer Token:   veryverysecret
Trickle ICE:    yes (HTTP PATCH)
Auto STUN/TURN: no
STUN server:    stun://stun.l.google.com:19302
TURN server:    (none)
Audio pipeline: (none)
Video pipeline: multifilesrc location=drone.mp4 loop=true ! qtdemux ! h264parse ! rtph264pay ! application/x-rtp,media=video,encoding-name=H264,payload=96

[WHIP] Initializing the GStreamer pipeline:
webrtcbin name=sendonly bundle-policy=0  stun-server=stun://stun.l.google.com:19302  multifilesrc location=drone.mp4 loop=true ! qtdemux ! h264parse ! rtph264pay ! application/x-rtp,media=video,encoding-name=H264,payload=96 ! sendonly. 
[WHIP] Starting the GStreamer pipeline

Hi

We tried to publish an RTSP stream using the simple-whip-client towards Millicast. It works, at least it publishes video to Millicast, but the playback is not smooth, a lot of framedrops, no smooth image, ... When trying the 'old' method, publishing using RMTP, playback is smooth (given the 3sec delay).

WHIP command:

docker run -it --rm -e "U=https://director.millicast.com/api/whip/STREAM" -e "T=TOKEN" -e "V=rtspsrc location=rtsp://192.168.30.3/Streaming/Channels/1 latency=0 name=rtsp ! rtph264depay ! rtph264pay config-interval=-1 ! application/x-rtp,media=video,encoding-name=H264" simple-whip-client

I ran the simple-whip-client on both RPI4 and on a X86 Mac OS, both using docker. The DockerFile entrypoint was changed a bit to be able to pass the TOKEN

ENV U=http://localhost:3000/whip/foo
ENV T=
ENV A="audiotestsrc is-live=true wave=red-noise ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay pt=100 ssrc=1 ! queue ! application/x-rtp,media=audio,encoding-name=OPUS,payload=100"
ENV V="videotestsrc is-live=true pattern=ball ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay pt=96 ssrc=2 ! queue ! application/x-rtp,media=video,encoding-name=VP8,payload=96"

ENTRYPOINT ./whip-client -l 7 -u $U -t $T -V "$V" -A "$A" -S stun://stun.l.google.com:19302

GStreamer command

gst-launch-1.0 \
rtspsrc location=rtsp://192.168.30.3/Streaming/Channels/1 latency=0 ! rtph264depay  ! flvmux name=mux ! queue ! rtmpsink location="rtmp://live-rtmp-pub.millicast.com:1935/v2/pub/STREAM?token=TOKEN live=1" 

EDIT Removed the error listening hacky code.

Now you pass in an extra param to give it a name of a sink between the source that may end by itself and webrtcbin Typically you're going to have a demuxer or a parser or a decoder anyway so just add a name to that "sink" and pass that name in as a param under -e

This is a small set of changes to add some preliminary support for the proposed ETag usages for PATCH, as documented in this early version of the updated WHIP draft: https://github.com/wish-wg/webrtc-http-ingest-protocol/pull/24/files?short_path=6ca9ae6#diff-6ca9ae63875a605dbaf6d8e45c08b5945951d62135ae0661065b7fe9ebb04d84

I made the same change in the WHIP server as well.

Notice that, since the WHIP client doesn't support ICE restarts yet, that part isn't integrated.

Probably don't need all those dependencies in the dockerfile... was a pretty rough and ready thing to get it to work. I'll come back and tidy it up as well as make other parameters controllable

You can already manually change the code to remove colours but you can't do it from the command line.

Also, if you turn off colours in the codebase, the [WHIP] prefix still has colors.

As the subject says, this PR adds a "no-trickle" mode: when enabled, the WHIP client will not trickle candidates via PATCH (default behaviour), but will wait for the gathering to complete to then add all candidates to the SDP offer instead. This shouldn't be needed even for servers that don't support trickle, as prflx candidates should still manage to get things working, but it's good to have the option anyway.

This is an attempt to address what's described in #4. As explained there, with GStreamer we can't add STUN/TURN servers after we've sent our offer, which means this code tries to do the same by using OPTIONS instead.

This feature is disabled by default: to tell the WHIP client to send the OPTIONS and get a list of STUN/TURN servers automatically, you need to pass the -f argument via command line. Notice that this will make the WHIP client ignore any other server you may have passed manually via -S and -T.

You can test this using the latest version of my WHIP server, which can be configured to return a list of STUN/TURN servers if you configure it accordingly (either statically in the configuration file, or per endpoint if you add them when creating a new one). I made a few tests and this seems to work as expected: of course more interop tests will need to be performed, as the code that processes the Link headers and turns them to addresses that can be used in GStreamer may be a bit flaky.

Adds support for multiple TURN servers, by adding multiple instances of the -T command line argument. Published as a pull request, rather than a direct commit, as I couldn't test this yet.

1024 was not enough on a host with both IPv4 and IPv6 addresses. Increase to 4096. We should probably be sending multiple fragments when we overflow.

The new version of the WHIP draft adds support for an automatic configuration of STUN/TURN servers to use in WHIP clients:

https://www.ietf.org/archive/id/draft-ietf-wish-whip-01.html#name-stun-turn-server-configurat

The way it works is that WHIP servers can return a series of STUN/TURN servers (and credentials, if needed) in Link headers in the 201 in response to the HTTP POST (or optionally in response to the OPTIONS too), e.g.:

Link: stun:stun.example.net; rel="ice-server";
Link: turn:turn.example.net?transport=udp; rel="ice-server"; username="user"; credential: "myPassword"; credential-type: "password";
Link: turn:turn.example.net?transport=tcp; rel="ice-server"; username="user"; credential: "myPassword"; credential-type: "password";
Link: turns:turn.example.net?transport=tcp; rel="ice-server"; username="user"; credential: "myPassword"; credential-type: "password";

The WHIP client can then parse those Link values and add the STUN/TURN servers to its own stack for gathering.

Unfortunately, this seems not to work as expected with GStreamer's webrtcbin. While there is, for instance, an add-turn-server signal that can be used to add a TURN server dynamically (which is what we now use for adding TURN servers at startup too, as implemented in #2), that doesn't seem to have any effect after the pipeline has been started. I tried delaying the set-local-description call to after we set the servers, and that didn't help.

Checking the code:

• add-turn-server calls gst_webrtc_bin_add_turn_server, which in turn calls gst_webrtc_ice_add_turn_server;
• gst_webrtc_ice_add_turn_server just adds the new server to a hashtable, ice->turn_servers.

The internal method that actually adds the TURN server to the libnice stack via nice_agent_set_relay_info, though, is _add_turn_server, which is only called by gst_webrtc_ice_add_stream. Since we add a stream when we first create our pipeline, and so always before we prepare an offer, trying to add a STUN/TURN server after that will not work.

The WHIP draft does have some text about this:

There are some webrtc implementations that do not support updating the ICE server configuration after the local offer has been created. In order to support these clients, the WHIP endpoint MAY also include the ICE server configuration on the responses to an authenticated OPTIONS request sent to the WHIP endpoint URL sent before the POST requests.

As such, I'll try to modify the WHIP server to also return a set of STUN/TURN servers in response to an OPTIONS on the endpoint URL, rather than just to the POST as I do now. This way, I can issue an OPTIONS request from the WHIP client before we create the pipeline, which should give us access to STUN/TURN servers we can actually use.