Summary:	ASTERISK-19353: musiconhold of remote party is replaced by local moh
Reporter:	Wolfgang Liegel (wliegel)	Labels:
Date Opened:	2012-02-14 05:55:28.000-0600	Date Closed:	2012-03-05 14:03:46.000-0600
Priority:	Minor	Regression?
Status:	Closed/Complete	Components:	Channels/chan_sip/General
Versions:	1.8.9.2	Frequency of Occurrence	Constant
Related Issues:
Environment:		Attachments:	( 0) asterisk_cli_sip_dump.txt
Description:	My Asterisk Server is connected via SIP Trunk to the PSTN. I place a call to a remote PBX (not an asterisk) or a mobile phone which places my call on hold. I'd expect to hear the remote moh but I receive my own asterisk's moh instead. This is a pure SIP environment. No DAHDI or IAX2.
Comments:	By: Wolfgang Liegel (wliegel) 2012-02-14 05:59:01.403-0600 I believe this issue is related to ASTERISK-15691 and ASTERISK-15258 and may also be related to ASTERISK-19095. By: Wolfgang Liegel (wliegel) 2012-02-14 06:10:44.506-0600 I did some research and SIP debugging: When a call is placed on hold by a regular SIP phone, the phone sends an SDP Body containing: c=IN IP4 0.0.0.0 a=sendonly But in this case, when my call is set on hold by the remote PBX (or mobile phone) I receive an SDP Body containing a valid IP in c= header: c=IN IP4 192.168.0.1 a=sendonly According to RFC 3264 (Section 6.1) the session should be modified to sendonly/recvonly mode without playing musiconhold. Putting a call on hold is described in Section 8.4 which also implies that the connection data c= is set to 0.0.0.0. By: David Woolley (davidw) 2012-02-14 06:22:26.314-0600 c=0.0.0.0 is an obsolete method. The problem with passing sendonly streams is that most phones use sendonly rather than inactive, to put the call on hold, but don't source MoH.  In fact, some equipment doesn't like receiving a=inactive. Incidentally, I believe that Asterisk should send a=sendonly when it is generating MoH, but we found that a modification we made to achieve this, and which was necessary in some contexts (outgoing REFER on some switches) caused problems with other switches (particularly Cisco). To not treat a=sendonly as a hold would, in my view, need to be treated as a new feature, and would need to be a per device option which defaulted to the current behaviour. By: Wolfgang Liegel (wliegel) 2012-02-14 06:24:24.871-0600 according to the asterisk CLI output my server starts musiconhold and (sometimes) reconnects the call: -- Started music on hold, class 'default', on SIP/123-00000006 -- Locally bridging SIP/123-00000006 and SIP/mysipprovider-00000007 this leads in Asterisk 1.8.9.2 to random behaviour: 1. sometimes the remote moh is played back as expected. 2. sometimes the local moh is played back. 3. sometimes both the local and remote moh are mixed together. This behaviour may change within a single SIP Session if the caller is placed on hold multiple times. By: Wolfgang Liegel (wliegel) 2012-02-14 06:41:25.794-0600 https://reviewboard.asterisk.org/r/1504/ looks to me like the c= header is being checked for 0.0.0.0 or :: . Isn't this information used to control the hold state? By: David Woolley (davidw) 2012-02-14 06:44:56.592-0600 The mixing of both MoH streams happens when you have direct media.  If the RTP flows through Asterisk, the upstream MoH is suppressed.  At least with some Cisco switches, the solution to this is to send a=sendonly with Asterisk's MoH, which will cause them to behave like Asterisk and source MoH downstream of Asterisk. Whilst I believe this is the correct thing to do when sourcing MoH, in practice, we had problems with this on some Cisco systems, but I have a feeling that this was a failure to produce the, correct, a=recvonly resonse to the re-invite. In any case, we are using a version of Asterisk with an optional new feature to generate a=sendonly when sending MoH and we have also reworked the delayed offer logic so that it works properly (Cisco's like using delayed offer SDP).  Unfortunately both of these have been done against 1.6.1.0 and there is little chance that we will have time to port them to trunk to enable them to be submitted. Basically, in the real world, SIP hold signalling and MoH injection is not straightforward, because the de facto modern method of signalling hold is to use a=sendonly, but terminals generally don't source music on hold when they do this, so the switch has to treat a=sendonly as a request to source MoH. By: David Woolley (davidw) 2012-02-14 07:02:29.840-0600 One could probably argue that Asterisk should be suspending direct media when it sources MoH.  That would avoid the mixing and random MoH selection problems. Especially with delayed offer, that would require another re-invite cycle after the incoming one to initiate the hold. By: David Woolley (davidw) 2012-02-14 07:05:35.846-0600 (Just for reference - I'm sure I've reported this elsewhere, the delayed offer problem is that Asterisk treats an incoming delayed offer as a request to go off hold, when it should actually not change state on the invite, but simply ensure that the OK sends its preferred state, rather than the negotiated state, as it would on the INVITE for an outgoing re-invite.) By: Wolfgang Liegel (wliegel) 2012-02-14 07:14:25.039-0600 All SIP peers are set to canreinvite=no but I get mixed streams nevertheless. Asterisk's behaviour when starting moh and not sending reINVITES is just fine. But in my case the asterisk server receives a reINVITE by the remote party which indicates that the remote party started moh. Unfortunately I can't affect the remote party's behaviour since this reINVITE comes from my SIP provider. But in my opinion there is way to distinguish a HOLD request from a sendonly reINVITE by checking the c= header being (not) null. This is not a delayed offer issue. The SDP comes with the (re)INVITE Request. By: David Woolley (davidw) 2012-02-14 07:31:47.174-0600 canreinvite is deprecated.  You should make sure that it is actually effective, or better just try with directmedia.  I don't think it has yet been withdrawn as an alias of directmedia, but I might be wrong. By: Wolfgang Liegel (wliegel) 2012-02-14 09:15:03.617-0600 I changed all peers to "directmedia=no" and verified all output from "sip show peer ...". After placing some calls I realize the behaviour isn't that random as I thought previously: A calls B. B puts A on hold (A gets B's moh - as expected). B gets A back and puts him on hold again (A gets mixed streams). if the call is placed on hold a 3rd time A gets B's moh only, because Asterisk can't find any more moh files: ast_moh_files_next: Unable to open file '/var/lib/asterisk/moh/file1': No such file or directory (I guess this should be placed in a different ticket since it doesn't match the description!?) I also tested this with 2 simple SIP Devices putting each other "really" on hold and moh was played without any errors each time. By: Matt Jordan (mjordan) 2012-02-20 10:53:53.621-0600 At this point in time, I'm confused as to what is actually being sent by what to what.  I think we're going to need a SIP pcap to determine why you're seeing what you're seeing. By: Wolfgang Liegel (wliegel) 2012-02-21 03:44:56.566-0600 I did a pcap on the client side to determine if asterisk sends multiple or one single mixed rtp stream. This pcap explained why I'm getting this "No such file or directory" error message. The client (eyeBeam 1.5.20.2) switches to g729 for no obvious reason. Asterisk tries to play moh but finds no codec translation path from g729 to ulaw which results in this error message. After disabling g729 on the client this error message disappears. This pcap also showed me that on the client side I receive multiple streams: one containing asterisk moh and one containing the remote moh. I tested this with different clients: e.g. Grandstream plays the asterisk moh stream until the remote moh is received while EyeBeam mixes both streams. Some other clients randomly play either asterisk or remote moh. I'll prepare a server-side pcap and post the results tomorrow. By: Wolfgang Liegel (wliegel) 2012-02-23 04:09:19.752-0600 I attached a SIP debug from asterisk cli also containing all started/stopped moh messages. As you can see the caller was put on hold twice since the issue with mixed streams did not occur during the first time the caller was put on hold. By: Wolfgang Liegel (wliegel) 2012-02-23 04:26:51.879-0600 I also did some Wireshark RTP Stream analyzing: In this call there are 14 different RTP Streams (8 different SSRC): 6x Client  <==> Asterisk <==> Provider 1x Asterisk --> Client  (asterisk moh) 1x Asterisk --> Provider The stream containing the asterisk moh is only sent to the client during the second onhold period. By: David Woolley (davidw) 2012-02-23 05:40:16.406-0600 Asterisk doesn't propagate the SSRC (which I consider a bug), which means that if you are seeing multiple SSRCs you have some direct media streams. By: Matt Jordan (mjordan) 2012-02-23 09:38:04.082-0600 As I mentioned previously, we really need the pcap to determine what's going on here. By: Wolfgang Liegel (wliegel) 2012-02-23 10:38:58.426-0600 The requested SIP pcap is already attached. I will not upload any pcap file containing real phone numbers or IP adresses. But if it helps I'd like to continue analyzing the data I've captured and provide you the results. By: Wolfgang Liegel (wliegel) 2012-02-24 02:26:23.372-0600 According to my RTP stream analyisis Asterisk does propagate the SSRC. The SSRC coming from my SIP provider is the same as the SSRC Asterisk sends to my client. On the Asterisk side I only see 2 different UDP ports for RTP communication (one to the client side and one to provider side). I guess, the reason why I see multiple SSRCs, is because the media session is modified by the SIP reINVITEs and therefore new SSRCs are created. By: David Woolley (davidw) 2012-02-24 05:41:24.272-0600 It might do so if packet to packet native bridging, but if there is a bridge up, there can be no music on hold. There is no ssrc field in the Asterisk frame structure, even in the trunk version. Packet to packet bridging is done in the technology handler module, bypassing the core of Asterisk, but MoH is handled by the core.  Of course, if Asterisk is failing to stop an MoH generator whilst packet to packet (or any other bridging mode) a call, that would be a bug. By: Wolfgang Liegel (wliegel) 2012-02-27 07:20:05.197-0600 There is no ssrc field in the Asterisk frame structure but in the Asterisk rtp structure (see rtp.h and struct ast_rtp), since SSRC is an essential part of the rtp header. As mentioned before I see multiple rtp streams which are obviously bridged between the client and the provider and I see 2 streams which are originated by Asterisk. The rtp stream which is sent to the client contains the moh from Asterisk. This moh stream is NOT being sent during the first time the call is on hold, but during the 2nd 3rd and so on. By: David Woolley (davidw) 2012-02-28 05:46:55.084-0600 That's why I said that packet to packet bridging may propagate it.  General bridging involves the receiving channel technology driver copying the rtp frame into an Asterisk frame and the bridge code regnerating the rtp frame from the Asterisk frame, thus losing the SSRC. My other point was, that whilst packet to packet bridging is running, any Asterisk tone generators are supposed to be shut down. By: Wolfgang Liegel (wliegel) 2012-02-28 07:23:10.942-0600 Sorry, I didn't know that there are different bridging modes in Asterisk. My post was founded on the debugging data I got from wireshark. So, since the SSRCs are propagated, I guess Asterisk is using packet to packet bridging mode and therefore Asterisk shouldn't send any moh, right? According to my wireshark analysis Asterisk is doing this right - but only at the first time the caller ist put on hold. So I guess this is a bug, isn't it? By: Matt Jordan (mjordan) 2012-03-05 14:03:38.933-0600 I'm sorry, but given the limited information I don't see a bug here at all.  From the little I can see in the CLI capture you attached, it appears as if two SIP channels are locally bridged, and that MOH is started and stopped at different points during that sequence.  The MOH appears to be starting and stopping based on the appropriate SDP's restricting/un-restricting of the media flow, which is what is expected. Since you've stated you won't upload the requested pcap showing the RTP streams you've been discussing, I'm not sure what else can be done on this issue.  As such I'm going to suspend this issue.  If you feel this is still a bug, please contact a bug marshall in #asterisk-bugs or start a discussion on the Asterisk users mailing list.