Summary:	ASTERISK-21172: One way audio when external Call forwarded to queue member
Reporter:	Peter Katzmann (pk16208)	Labels:
Date Opened:	2013-02-27 02:05:55.000-0600	Date Closed:	2013-07-15 16:10:02
Priority:	Major	Regression?
Status:	Closed/Complete	Components:	Applications/app_queue Channels/chan_local Channels/chan_sip/General
Versions:	SVN 1.8.20.1 1.8.21.0 1.8.23.0	Frequency of Occurrence	Constant
Related Issues:
Environment:	Ubuntu 10.04LTS, I686pae	Attachments:	( 0) caller-to-callee.pcap ( 1) caller-to-markus-from-asterisk-via-patton.pcap ( 2) call-from-patton-to-callee-asterisk.pcap ( 3) digium-info_20130226.tar.gz ( 4) flow-queue ( 5) flow-user ( 6) from-asterisk-to-queue1817.cap ( 7) from-asterisk-to-queue-member.pcap.bz2 ( 8) from-asterisk-to-queuemember-1817 ( 9) from-ext-to-asterisk.pcap.bz2 (10) from-or-to-asterisk-patton5.pcap.bz2 (11) from-patton-to-asterisk-1817 (12) from-patton-to-asterisk-1817-2.pcap (13) hotline.log.bz2 (14) logs-1921-v2-patch.tar.bz (15) no-audio.log (16) no-audio.pcap (17) oneWayAudio-1.8.21 (18) onewayaudio-1.8.21.pcap (19) oneWayAudio-svn (20) onewayaudio-svn.pcap (21) pcaps-1921-v2-patch.tar.bz (22) withandwithout.pcap
Description:	When i call from external an colleague and he is not available the call is forwarded to the hotline queue. On queue operator pick up the call, ringing stops on caller side ba he can't here the agent but the agent can hear the caller. When i try the same scenario internal audio ist ok Asterisk 1.8.17 is working fine Core dump during one call is available so if there is some information inside give me some information how to retrieve them.
Comments:	By: Peter Katzmann (pk16208) 2013-02-27 02:41:02.128-0600 Core dump is also available as a snapshot made during the call but to large to upload By: Rusty Newton (rnewton) 2013-03-01 10:56:04.895-0600 Peter can you provide PCAPs for both the working (1.8.17?) and non working 1.8.20.1 scenarios? By: Rusty Newton (rnewton) 2013-03-01 10:59:38.466-0600 Oh! and provide the full log for both those calls you gather PCAPs for, including VERBOSE and DEBUG turned up to level 5 (you had only verbose in the attached files) By: Peter Katzmann (pk16208) 2013-03-05 03:07:25.247-0600 Call from asterisk to queue member By: Peter Katzmann (pk16208) 2013-03-05 03:07:51.042-0600 from external mobile phone to asterisk By: Peter Katzmann (pk16208) 2013-03-05 03:10:37.136-0600 Hello Rusty, debug log for 1.8.20.1 will take till next week, because i need the live system and switching forward and back during business hours will be a no go. Trace of 1.8.17 (with debug) will be follow soon peter By: Rusty Newton (rnewton) 2013-03-27 17:03:57.985-0500 Peter, which pcap corresponds to the calls in the "onewayAudio" file? We need the corresponding pcap to really see what is happening here. By: Peter Katzmann (pk16208) 2013-03-28 02:45:00.058-0500 Hello the call from 00151 is the incoming mobile phone The destination was the 17059 forwarded to the queue member 5333 By: Peter Katzmann (pk16208) 2013-04-02 09:55:44.180-0500 pcap file to debug file By: Peter Katzmann (pk16208) 2013-04-02 09:55:57.316-0500 pcap is attached By: Rusty Newton (rnewton) 2013-04-04 17:17:11.970-0500 Peter, Does this only happen with the one queue member 5333 ? Does this only occur with the one particular external caller? Does this happen with any caller, regardless of which queue member they get transferred too? For the caller or queue member this happens with - does it happen on *every* call or is it random? Is there a chance you could test with the latest SVN snapshot of the 1.8 branch? (and report which revision you tried) Can you provide an example PCAP, along with the Asterisk log file (DEBUG,VERBOSE) and SIP debug enabled? The SIP debug in the Asterisk log helps to correlate what's happening in the PCAP to the Asterisk log. If you can help to narrow down the most minimal configuration it would require to reproduce this issue - that would be extremely useful. I'm having issues debugging your packet captures - wireshark doesn't like some of the RTP and is crashing. If we can reproduce the issue that would be easier. By: Rusty Newton (rnewton) 2013-04-04 17:25:46.459-0500 Another question! Which call in the onewayaudio.pcap is supposed to have one way audio? The first or second? Be sure to hit "Send Back" after you respond. By: Peter Katzmann (pk16208) 2013-04-05 03:06:41.315-0500 Does this only happen with the one queue member 5333 ? -> NO WITH EVERY QUEUE MEMBER Does this only occur with the one particular external caller? -> NO WITH ALL EXTERNAl CALLER Does this happen with any caller, regardless of which queue member they get transferred too? -> NO ONLY WITH EXTERNAL CALLS FORWARDED FROM AN INTERNAL USER TO THE QUEUE For the caller or queue member this happens with - does it happen on every call or is it random? -> IF IT IS A EXTERNAL CALL FORWARDED OVER AN INTERNAL USER YES, ON EVERY CALL Is there a chance you could test with the latest SVN snapshot of the 1.8 branch? (and report which revision you tried) -> WELL NOT SO EASY AND IT COULD TOOK A FEW DAYS Can you provide an example PCAP, along with the Asterisk log file (DEBUG,VERBOSE) and SIP debug enabled? The SIP debug in the Asterisk log helps to correlate what's happening in the PCAP to the Asterisk log. -> FOR THIS A HAVE TO GENERATE A NEW ONE, BUT THIS WILL TOOK SOME TIME Which call in the onewayaudio.pcap is supposed to have one way audio? The first or second? -> IT SHOULD BE ON BOTH I will try to build a lap system to simulate the problem By: Rusty Newton (rnewton) 2013-04-08 14:39:52.897-0500 deleted a duplicate comment By: Rusty Newton (rnewton) 2013-04-08 15:14:52.097-0500 {quote} I will try to build a lap system to simulate the problem {quote} Thanks - if you can provide a step by step guide for reproduction that would make this much clearer and easier to pinpoint the issue. By: Rusty Newton (rnewton) 2013-04-09 14:50:52.697-0500 Just adding that some others here looked over your onewayaudio.pcap and couldn't identify any one-way audio issues. A reproduction guide is still the next best step.  Thanks! By: Peter Katzmann (pk16208) 2013-04-10 07:12:01.532-0500 Hello, now if finished a lab installation and can also reproduce the one way audio. Tested with svn current branch checkout and release 1.8.21 Call Flow: User 1003 calls 1001 1001 does not pick up the call call will be forwarded to queue number 05 (queue 75111) after 5 seconds member 1000 in queue is ringing member 1000 pickup the call member 1000 and 1003 get connected information audio from member 1000 to 1003 is not heard audio from member 1003 to 1000 can be heard Direct call from 1000 to 1003 and vice versa is working files : onewayaudio-1812 and onewayaudio-svn By: Peter Katzmann (pk16208) 2013-04-10 07:13:43.882-0500 Files to report from 10.04.2013 By: Rusty Newton (rnewton) 2013-04-17 16:33:58.298-0500 So, as far as the PCAP goes, I can't identify a one-way audio issue. That is - I see RTP to and from endpoints on both sides of Asterisk (from 1003 to asterisk to 1000 and 1000 to 1003). The RTP is not silence, it contains audio that a user should recognize (ringing, blowing in handset, etc) In the accompanying logs I don't see anything obviously wrong, as in  any unusual errors or debug that would indicate an obvious issue. I'd say that the "Server: OpenStage_60_V3 R0.73.0" is likely receiving the RTP, but possibly dropping it for one reason or another.. Your next step is to look at the log on the Siemens unit and interpret what it is doing with the received RTP and then let us know. If there is a bug here, then it would mean that Asterisk is doing something goofy with the RTP and the Siemens unit doesn't like it. Otherwise this may just be a misconfiguration on the Siemens side of things. Summary: * Lack of any errors or unusual behavior in logs shows that Asterisk doesn't know that it is doing anything wrong. * A cursory analysis of the PCAP doesn't show anything obviously wrong. * You are saying that a user behind the endpoint at 10.0.0.100 in your PCAP isn't hearing anything so we need to know: ** Does that endpoint (OpenStage) think something is wrong with the RTP it is receiving from Asterisk? ** What does the endpoint (OpenStage) think is wrong with the RTP? * Alternatively you may swap out the OpenStage endpoint with a any other SIP phone (Polycom or even Digium would be easy for us to reproduce with if you can show it happens with those) At this point this doesn't appear to be a bug. I'll leave this in Waiting on Feedback for a couple weeks to give you a chance to look at the Siemens. By: Peter Katzmann (pk16208) 2013-04-26 08:34:48.025-0500 OK, trace and pcap, caller calls 01 forwarded after 05 seconds to queue. queue member picks up very fast. Two response, queue member can talk and hear. Caller can talk, but can hear nothing or sometimes her the ringtone By: Rusty Newton (rnewton) 2013-04-29 16:11:56.468-0500 I still don't see a clear issue or difference in the new captures. You may also want to rephrase what you intended to demonstrate with the new captures, as it is unclear. In relation to another issue ASTERISK-21246. Can you try setting "rtpkeepalive" to "rtpkeepalive=0" ? I know the reporter of that issue had audio problems with Snom phones. Also you might leave it on and try the latest patch from that issue: https://issues.asterisk.org/jira/secure/attachment/47187/asterisk-21246-rtp-cng-payload-error_1.8_v2.diff By: Rusty Newton (rnewton) 2013-04-29 16:14:54.388-0500 Haha. I just noticed that you are the reporter of ASTERISK-21246 as well. I must be sleepy! Still, please confirm whether for this particular scenario you have disabled rtpkeepalive or tried the patch? By: Peter Katzmann (pk16208) 2013-04-30 03:41:56.478-0500 Ok, now traces with the v2 rtp patch. The szenario is: A calls B B has forwarding to queue C where D is member of it. Now when D pickup the call relatively fast/during first ring: A does hear nothing from D or it is still ringing D can hear A When A calls directly the queue everything is fine. I now made traces with several manufacturers as caller and callee During analysis of the pcap it seems that A get's the audio delivered but only after D put the call on hook. It's in all calls the same Hope this clarify this situation a little bit By: Rusty Newton (rnewton) 2013-05-10 16:32:20.752-0500 Thanks for that. I still see RTP being received by Asterisk and sent out by Asterisk on both call legs. The audio going outbound from Asterisk to the original caller(or "A" in your above description) is able to be decoded and played in wireshark. I'm not sure if anything is wrong with the RTP in such a way that would cause the receiving phone to ignore or discard it. I'll have someone with more advanced RTP knowledge take a look at it to double check. Can you reproduce this issue when having B forward A to another extension, vs forwarding to a queue? By: Peter Katzmann (pk16208) 2013-05-13 07:33:01.720-0500 When i forward the call directly i have no problems. It's only with queue member In the attached sample: First call picked up fast and caller can still hears ringing (hearable in the file) Second call i wait one complete ring and then pick up the call and audio is ok, is hearable as feedback noise in file. By: Matt Jordan (mjordan) 2013-07-15 16:09:48.346-0500 Unfortunately, I'm not able to reproduce this problem. I did the following to reproduce this issue: * Three phones - {{digium01}}, {{digium02}}, and {{phoneb}} - were used. {{phoneb}} is the caller; {{digium01}} is the SIP phone that forwards the call; {{digium02}} is the queue member * {{phoneb}} dials extension 1000, which dials {{digium01}} * {{digium01}} forwards the request to extension 5000 * Extension 5000 puts {{phoneb}} into the {{sales}} queue * A Local channel member dials {{digium02}} * {{digium02}} Answers * Both {{phoneb}} and {{digium02}} can converse. There were no one-way audio issues. Sample configurations: {noformat:title=extensions.conf} [default] exten => 1000,1,NoOp()        same => n,Verbose(1, Dialing digium01)        same => n,GoSub(default,set_caller_id,1)        same => n,Dial(SIP/digium01,15,hHkKtT)        same => n,Hangup() exten => 5000,1,NoOp()        same => n,Queue(sales) [agents] exten => digium02,hint,SIP/digium02 exten => digium02,1,NoOp()        same => n,Dial(SIP/digium02,15,hHkKtT)        same => n,Hangup() {noformat} {noformat:title=sip.conf} [peer-template](!) directmedia = yes disallow = all allow = g722 allow = gsm allow = ulaw allow = alaw [phone_b](peer-template) type = peer secret = directmedia = no callerid = "Phone B" <B> host = dynamic context = default sendrpid=pai mailbox=3000 [digium02](peer-template) type = peer secret = digium02 directmedia = no callerid = "Digium 02" <102> host = dynamic context = default sendrpid=pai setvar=AGENTNAME=digium02 mailbox=2000 [digium01](peer-template) type = peer secret = digium01 directmedia = no callerid = "Digium 01" <101> host = dynamic context = default sendrpid=pai mailbox=1000 setvar=AGENTNAME=digium01 {noformat} {noformat:title=queues.conf} [sales] strategy=ringall announce=sales musicclass = default monitor-type=MixMonitor monitor-format=wav member => Local/digium02@agents {noformat} Below is a dump of the CLI during execution: {noformat}    -- Executing [1000@default:1] NoOp("SIP/phone_b-00000003", "") in new stack    -- Executing [1000@default:2] Verbose("SIP/phone_b-00000003", "1, Dialing digium01") in new stack  Dialing digium01    -- Executing [1000@default:3] Gosub("SIP/phone_b-00000003", "default,set_caller_id,1") in new stack    -- Executing [set_caller_id@default:1] NoOp("SIP/phone_b-00000003", "") in new stack    -- Executing [set_caller_id@default:2] Set("SIP/phone_b-00000003", "CALLERID(name)=Asterisk") in new stack    -- Executing [set_caller_id@default:3] Set("SIP/phone_b-00000003", "CALLERID(num)=555-5555") in new stack    -- Executing [set_caller_id@default:4] Return("SIP/phone_b-00000003", "") in new stack    -- Executing [1000@default:4] Dial("SIP/phone_b-00000003", "SIP/digium01,15,hHkKtT") in new stack  == Using SIP RTP CoS mark 5    -- Called SIP/digium01    -- Got SIP response 302 "Moved Temporarily" back from 10.24.19.80:5060    -- Now forwarding SIP/phone_b-00000003 to 'Local/5000@default' (thanks to SIP/digium01-00000004) [Jul 15 15:51:15] NOTICE[17699]: app_dial.c:901 do_forward: Not accepting call completion offers from call-forward recipient Local/5000@default-00000003;1    -- Executing [5000@default:1] NoOp("Local/5000@default-00000003;2", "") in new stack    -- Executing [5000@default:2] Queue("Local/5000@default-00000003;2", "sales") in new stack    -- Started music on hold, class 'default', on Local/5000@default-00000003;2    -- Executing [digium02@agents:1] NoOp("Local/digium02@agents-00000004;2", "") in new stack    -- Executing [digium02@agents:2] Dial("Local/digium02@agents-00000004;2", "SIP/digium02,15,hHkKtT") in new stack  == Using SIP RTP CoS mark 5    -- Called SIP/digium02    -- Local/digium02@agents-00000004;1 connected line has changed. Saving it until answer for Local/5000@default-00000003;2    -- Local/digium02@agents-00000004;1 connected line has changed. Saving it until answer for Local/5000@default-00000003;2    -- SIP/digium02-00000005 is ringing    -- Local/digium02@agents-00000004;1 is ringing    -- Local/digium02@agents-00000004;1 connected line has changed. Saving it until answer for Local/5000@default-00000003;2    -- SIP/digium02-00000005 answered Local/digium02@agents-00000004;2    -- Local/digium02@agents-00000004;1 answered Local/5000@default-00000003;2    -- Stopped music on hold on Local/5000@default-00000003;2  == Begin MixMonitor Recording Local/5000@default-00000003;2    -- Local/5000@default-00000003;1 answered SIP/phone_b-00000003    -- Executing [h@default:1] NoOp("Local/5000@default-00000003;2", "") in new stack    -- Executing [h@default:2] Wait("Local/5000@default-00000003;2", "10") in new stack  == Spawn extension (default, h, 2) exited non-zero on 'Local/5000@default-00000003;2'  == Spawn extension (default, 5000, 2) exited non-zero on 'Local/5000@default-00000003;2'  == Spawn extension (agents, digium02, 2) exited non-zero on 'Local/digium02@agents-00000004;2'    -- Executing [h@default:1] NoOp("SIP/phone_b-00000003", "") in new stack    -- Executing [h@default:2] Wait("SIP/phone_b-00000003", "10") in new stack  == Spawn extension (default, h, 2) exited non-zero on 'SIP/phone_b-00000003'  == MixMonitor close filestream  == End MixMonitor Recording Local/5000@default-00000003;2  == Spawn extension (default, 1000, 4) exited non-zero on 'SIP/phone_b-00000003' {noformat} I'm not sure what else we can do on this issue. We cannot reproduce it, nor can we determine from your pcaps what the issue is. As Rusty mentioned, in the {{no-audio.pcap}} RTP appears to be flowing at all times between all three participants. From Asterisk's perspective, it appears as if RTP is going back and forth between the phones. Since we can't reproduce this issue, I'm going to go ahead and close it out as "Can't Reproduce". If you can determine what in the configuration is different between my attempt at reproducing the problem and your lab set up and can suggest how we can reproduce the problem, we can reopen this issue. By: Peter Katzmann (pk16208) 2013-08-14 09:13:29.150-0500 Just some new Log files because the Problem still persist with 1.8.23 You can hear clearly nothing on the caller side. Hope this will help to clear the case. What i don't understand is that the callee rtp stream changes the ssrc and ports during call. Revert back to 1.8.17 and everything works as expected Please examine this again By: Rusty Newton (rnewton) 2013-09-03 18:25:53.123-0500 Peter, we have many issues to look at on a daily basis. We unfortunately don't have the time to continue looking at this issue when we can't reproduce it or see anything obviously wrong. If you can describe exactly how to reproduce the issue then we could look at re-opening it. I'd recommend providing paste-bins and packet captures to the asterisk-users list and asking for help. You can always of course point them here to look at the information you have already provided. Since we couldn't find an issue with the traffic or Asterisk before, the issue may lie with your endpoints or networking specifics. Those are always difficult to track down.