[Home]

Summary:ASTERISK-24735: [patch] - Video Media support broken for (WebRTC endpoints)
Reporter:Javier Riveros (goseeped)Labels:patch
Date Opened:2015-01-28 14:22:14.000-0600Date Closed:
Priority:MajorRegression?Yes
Status:Open/NewComponents:Channels/chan_pjsip Channels/chan_sip/SRTP Channels/chan_sip/Video Resources/res_srtp
Versions:12.8.0 13.1.0 13.1.1 13.2.0 13.7.2 Frequency of
Occurrence
Constant
Related
Issues:
Environment:OS: ubuntu 14.04 Asterisk: 13.1.0 current version. Client : jssip 0.6.12 online demo ( disable new session timers feature ) /firefox 34 /chrome 39 channels : Chan_sip , chan_pjsipAttachments:( 0) Ast_Debug_WebRTC-VP8LOG
( 1) firefox_debug_output.txt
( 2) frame.c.diff
( 3) res_rtp_asterisk.c.diff
( 4) rtp_ast_13_1_vp8_error.txt
( 5) rtp_conf.txt
( 6) sip_ast_13_1_vp8_error.txt
( 7) sip_conf.txt
Description:If this is a duplicate: sorry for the noise. I failed to find it on this versions.

Test ) Call between two webrtc peers firefox 34 jssip client, asterisk playback audio before dial.

On Playback(letters/asterisk); works great
On Dial ;dial works for chan_sip or chan_pjsip. i only get this warnings
{noformat}
WARNING[10405][C-00000001]: res_srtp.c:407 ast_srtp_unprotect: SRTP unprotect failed with: authentication failure 10
WARNING[10405][C-00000001]: res_srtp.c:407 ast_srtp_unprotect: SRTP unprotect failed with: authentication failure 110
{noformat}

Results

- Audio ulaw works great.
- Video VP8 not work on ast 13.1 ; in  ast 11 with patch VP8 pass.. video work with same versions of clients and configs,.
- Signalling seems to be OK. compare with ast11
- This behavior is the same for chan_sip and chan_pjsip.
- Curious thing if you call from softphones (linphone) that support udp vp8 to web browser , linphone could see video from web browser but web browser couldn't see video from linphone, seems like asterisk is changing something on VP8 streams when webrtc peer is involve.

When you call between WebRTC endpoins Asterisk 13.1 is sending media (audio, video) to both legs of the call but video part not work , Firefox/chrome Video debug said "Received incomplete frame timestamp" and "Decoder error: -1"

{noformat}
DEBUG     ; (15: 1:23:345 |    0) VIDEO CODING:    0     1;      8259; Delay: min_playout=0 jitter=96 max_decode=0 render=10
DEBUG     ; (15: 1:23:346 |    1) VIDEO CODING:    0     1;      8259; ExtrapolateLocalTime(1357020)=22163368 ms
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; Render frame 3159780769 at 1357020. Render delay 22163474
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; Delay: min_playout=0 jitter=96 max_decode=0 render=10
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; Received incomplete frame timestamp 1353960 frame size 809 at time 22163413
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; Packet received and sent to jitter estimate with: timestamp=1353960 wall_clock=22163413
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; Jitter estimate updated with: frameSize=809 frameDelayMS=-5
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; Framesize statistics: max=1870.187082 average=1469.215404
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; The estimated slope is: theta=(0.002510, 10.590306)
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; Random jitter: mean=-2.833788 variance=2419.359935
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; Current jitter estimate: 85.612250
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; Current max RTT: 0
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; g1=0.000000 g2=-384.621773 alarm=0
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; w[0]=89.480924 w[1]=-238058.950884 ts=1357020 tMs=17553
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     1;      8259; Delay: min_playout=0 jitter=96 max_decode=0 render=10
DEBUG     ; (15: 1:23:346 |    0) VIDEO CODING:    0     0;      8259; Decoding timestamp 1357020
ERROR     ; (15: 1:23:346 |    0) VIDEO CODING:    0     0;      8259; Decoder error: -1
ERROR     ; (15: 1:23:346 |    0) VIDEO CODING:    0     0;      8259; Failed to decode frame 1357020, requesting key frame
{noformat}

Update 29 Jan 2015: this happens for h264 codecs to.

Thanks
Comments:By: Javier Riveros (goseeped) 2015-01-28 14:25:14.209-0600

Logs

By: James Van Vleet (jvanvleet) 2015-02-03 15:13:26.605-0600

I found the issue with video not working with Chrome and Firefox.    The marker field in the RTP header is not being passed though the bridge properly for video frames.  I have attached a patch that fixes the issue.  It might be more proper to move the fix farther up to where the marker is pulled out of the packet, however I was not sure I would not break it in all the places it is used for audio so I will leave that up to the experts.

I signed a license agreement and had it accepted long ago, but I signed another one just in case it is not linked up properly.  Patch attached.

By: James Van Vleet (jvanvleet) 2015-02-04 16:12:58.410-0600

Yeah gonna need an adult over here.  :-)

Javier pointed out that I only fixed video using applications (such as echo()) with the rtp patch.   I traced it back and the frame_ending bit was being lost on ast_frdup as well (called from softmix).  Second patch fixes the issue but I am not an expert enough to say if persisting frame_ending is cool for all voice, video and text frames or if it should be narrowed to video only.

By: Javier Riveros (goseeped) 2015-02-16 12:01:37.597-0600

Oh yes a little of help here would be good this is an interesting feature that asterisk 13 definitely should support.

@James after your second patch I was able to :

- We are able to make direct calls between Webrtc Endpoints in some conditions (in some clients and web browser doesn't work).
- We notice after second patch when video is involved on Webrtc endpoints that Playback of audio files doesn't work either.

if you need some extra information let me know.

By: Matt Jordan (mjordan) 2015-02-16 12:58:26.524-0600

[~goseeped]:

You need to provide more context for your statements.

{quote}
* We are able to make direct calls between Webrtc Endpoints in some conditions (in some clients and web browser doesn't work).
{quote}

Specifically, what worked? What didn't work? What channel driver were you using? What browsers?

{quote}
* We notice after second patch when video is involved on Webrtc endpoints that Playback of audio files doesn't work either.
{quote}

That's going to be an audio stream issue, and unrelated to the video stream.

[~jvanvleet]:

Setting the {{frame_ending}} integer to non-zero is appropriate for video frames, and shouldn't affect non-video streams:

{code}
struct ast_frame_subclass {
/*! A frame specific code */
int integer;
/*! The asterisk media format */
struct ast_format *format;
/*! For video formats, an indication that a frame ended */
unsigned int frame_ending;
};
{code}

So I'd suspect that your solution is correct, or at least very close to being completely correct.

By: Jacek Konieczny (jkonieczny) 2016-03-22 05:03:15.231-0500

Any progress on Video support with WebRTC?

Is there a chance the problem is related to the one described on the mailing list: http://lists.digium.com/pipermail/asterisk-users/2016-February/288392.html ?

By: Javier Riveros (goseeped) 2016-03-22 09:18:34.516-0500

Thats exactly what we were experience at the time we try to setup a video call in 13.2 with the previous patch now seems like 13.7 doesn't need the provious patch anymore but still has the video delay due the following statement.

{quote}
this is due to the time needed for the DTLS
handshake between Asterisk and the caller. Since Asterisk first
establishes a full connection to the callee, the callee already begins
sending data, while Asterisk is still performing the DTLS handshake with
the caller. As a consequence the caller misses the first RTCP Full
Intraframe Request (FIR) and the received video stream cannot be
rendered till the next FIR 90s later arrives.
{quote}

By: Jacek Konieczny (jkonieczny) 2016-03-24 10:14:19.174-0500

I think the video problems can be better explained by ASTERISK-25642