[Home]

Summary:ASTERISK-28661: chan_iax jitterbuffer growing when time sources not in sync
Reporter:Kirsty Tyerman (ktyerman)Labels:
Date Opened:2019-12-15 23:35:25.000-0600Date Closed:
Priority:MajorRegression?
Status:Open/NewComponents:Channels/chan_iax2
Versions:16.4.0 Frequency of
Occurrence		Constant
Related
Issues:
Environment:Attachments:( 0) adjust_clock.sh
( 1) debug_log_28661
( 2) iax.conf
( 3) iax2_show_channels.txt
Description:I have two asterisk servers, Server A and Server B. These two servers are peered via IAX. When I make a call between Server A and Server B I am experiencing poor audio quality and the jitter buffer on one end is continuously growing.
Server A's time is approximatley 10 minutes behind Server B.
I have a PJSIP client registered to Server B call a PJSIP client registered to Server A. The call is established via IAX between Server A and Server B.
I have the IAX jitter buffers enabled on both Server A and Server B with trunktimestamps=yes.
When I establish the call, the "iax2 show channels" on Server A (date is behind) shows the "JitBuf" constantly growing. The max jitter buffer is 4000ms. It seems as though the jitter buffer is not handling the system clock being wrong when it should rely on the trunk timestamps that is relative to iax and disregard any system clocks.
Comments:By: Asterisk Team (asteriskteam) 2019-12-15 23:35:27.126-0600

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.
By: Kevin Harwell (kharwell) 2019-12-17 18:14:57.494-0600

Please post the output from
{noformat}
*CLI> iax2 show channels
{noformat}
As well please enable general debugging[1] in the Asterisk log (at least level 2), iax debugging, and jitterbuffer debugging. Jitterbuffer debugging can be enabled by issuing the following on the CLI:
{noformat}
*CLI> iax2 set debug jb on
{noformat}
Note you can disable the jitterbuffer debugging using the following:
{noformat}
*CLI> iax2 set debug jb off
{noformat}
Then attach the full Asterisk log to this issue.

Also please attach your _iax.conf_, or at least the relevant portions from it.

[1] https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information

Thanks!


By: Kirsty Tyerman (ktyerman) 2019-12-19 18:47:27.688-0600

Please find attached:
* iax.conf: the iax conf file that was used
* iax2_show_channels: a watch of "iax2 show channels" over the duration of a call and you can see that the jitter buffer is constantly increasing.
* debug_log_28661: the log file whit debugging turned on where you can see the "G" indicates the jitter buffer is constantly growing.

There is no jitter between the two asterisk servers.

The timing source selected by asterisk is "res_timing_dahdi.so".

The system is running chrony, which is adjusting slowly the system clock to the NTP server. When I stop chrony from adjusting the clock, there is no longer an issue, even if the dates are not synced. Seems to be a bug with how asterisk jitter buffer is referencing time. It is stated in the "IAX2 Jitterbuffer" that "Different clock rates are handled very gracefully though; it will actually deal with a sender sending 20% faster or slower than you expect just fine." Looking at the asterisk sources there is reference to timer_t, which is the system clock.

Asterisk is not handling gracefully the system clock being slewed, as the jitter buffers are unnecessarily growing and this is affecting voice quality.
By: Kevin Harwell (kharwell) 2020-01-03 09:51:06.533-0600

I was able to replicate this issue using the given iax config, and the attached [^adjust_clock.sh] script I wrote. I used the same configuration for both Asterisk instances (changing the host of course, and disabled authentication). I started the script, and then orginated a call using the iax endpoint from one Asterisk instance to the other. For instance:
{noformat}
*CLI> channel originate IAX2/audio@<other Asterisk host/ip>/940 extension 940
{noformat}
Extension 940 is just a simple answer and playback:
{noformat}
exten => 940,1,NoOp()
same => n,Answer()
same => n,Playback(demo-congrats)
same => n,Hangup()
{noformat}
The [^adjust_clock.sh] script increases the current system time by one second every 3 seconds, and then stops after 10 times. **Note** it does not automatically reset/resync the system time. After running it you'll have to manually do that yourself with whatever system command works for your system. For instance, I was able to sync the clock on the machine (Ubuntu 14.04) I was using to test by executing the following:
{noformat}
$ sudo timedatectl set-ntp off
$ sudo timedatectl set-ntp on
{noformat}
By: Kevin Harwell (kharwell) 2020-01-03 10:11:18.970-0600

[~ktyerman] We're going to open this issue, but want to set some expectations.

I believe this issue could be interpreted as an improvement. I think it falls more under the "Bad timestamps" category where the sender is sending bad data. That said according to the wiki [1] we could be a bit more robust in handling these types of situations. So really this issue kind of spans the border of bug and improvement.

Not sure of a good workaround at this time. Perhaps lowering the _maxjitterbuffer_ size a bit?

Just a note too, but in my testing if I slewed the clock more than a few seconds the buffer would automatically resync and the problem did not occur. So the window for this isn't large.

[1] https://wiki.asterisk.org/wiki/display/AST/IAX2+Jitterbuffer