| Summary: | ASTERISK-28415: segfault: sprint_list_entry (entry=entry@entry=0x7f9e30b4d8b0, line=line@entry=0x7f9e70676590 "\340ggp\236\177", len=256) at res_pjsip_history.c:669 | ||||
| Reporter: | Brian J. Murrell (brian_j_murrell) | Labels: | pjsip | ||
| Date Opened: | 2019-05-14 06:00:46 | Date Closed: | 2020-08-28 11:34:07 | ||
| Priority: | Major | Regression? | |||
| Status: | Closed/Complete | Components: | pjproject/pjsip | ||
| Versions: | 13.26.0 | Frequency of Occurrence | Constant | ||
| Related Issues: |
| ||||
| Environment: | Attachments: | ( 0) ThreadStacktrace.txt | |||
| Description: | When I turn on pjsip history I get segfaults. The thread that faults is:
{noformat} #0 0x00007f9e507f736a in sprint_list_entry (entry=entry@entry=0x7f9e30b4d8b0, line=line@entry=0x7f9e70676590 "\340ggp\236\177", len=256) at res_pjsip_history.c:669 #1 0x00007f9e507f8435 in history_on_rx_msg (rdata=<optimized out>) at res_pjsip_history.c:763 #2 0x00007f9e9cd94167 in pjsip_endpt_process_rx_data (endpt=endpt@entry=0x27cbf88, rdata=rdata@entry=0x7f9e90096e78, p=p@entry=0x7f9e70676780, p_handled=p_handled@entry=0x7f9e70676770) at ../src/pjsip/sip_endpoint.c:893 #3 0x00007f9e9cd9432e in endpt_on_rx_msg (endpt=0x27cbf88, status=<optimized out>, rdata=0x7f9e90096e78) at ../src/pjsip/sip_endpoint.c:1043 #4 0x00007f9e9cd9b9c2 in pjsip_tpmgr_receive_packet (mgr=<optimized out>, rdata=rdata@entry=0x7f9e90096e78) at ../src/pjsip/sip_transport.c:2026 #5 0x00007f9e9cd9ddd0 in udp_on_read_complete (key=0x7f9e70a8da40, op_key=<optimized out>, bytes_read=869) at ../src/pjsip/sip_transport_udp.c:191 #6 0x00007f9e9ce14ffc in ioqueue_dispatch_read_event (ioqueue=ioqueue@entry=0x7f9e70a8d0a0, h=h@entry=0x7f9e70a8da40) at ../src/pj/ioqueue_common_abs.c:605 #7 0x00007f9e9ce16710 in pj_ioqueue_poll (ioqueue=0x7f9e70a8d0a0, timeout=timeout@entry=0x7f9e70676d40) at ../src/pj/ioqueue_epoll.c:720 #8 0x00007f9e9cd93e98 in pjsip_endpt_handle_events2 (endpt=0x27cbf88, max_timeout=max_timeout@entry=0x7f9e70676d90, p_count=p_count@entry=0x0) at ../src/pjsip/sip_endpoint.c:744 #9 0x00007f9e9cd93f57 in pjsip_endpt_handle_events (endpt=<optimized out>, max_timeout=max_timeout@entry=0x7f9e70676d90) at ../src/pjsip/sip_endpoint.c:776 #10 0x00007f9e7b5ae628 in monitor_thread_exec (endpt=<optimized out>) at res_pjsip.c:4512 #11 0x00007f9e9ce17ae0 in thread_main (param=0x27cb7b8) at ../src/pj/os_core_unix.c:541 #12 0x00007f9e9b3b5dd5 in start_thread (arg=0x7f9e70677700) at pthread_create.c:307 #13 0x00007f9e9a986ead in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 {noformat} | ||||
| Comments: | By: Asterisk Team (asteriskteam) 2019-05-14 06:00:47.712-0500 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur. By: Brian J. Murrell (brian_j_murrell) 2019-05-14 06:01:28.207-0500 [Threaded stack trace|https://issues.asterisk.org/jira/secure/attachment/58348/ThreadStacktrace.txt] By: George Joseph (gjoseph) 2019-05-15 08:08:38.186-0500 Hmmm. That's a weird place to get a segfault. Does it seem to happen with IPv6 or IPv4 as well? Can you give me another backtrace so I can compare the two? By: Brian J. Murrell (brian_j_murrell) 2019-05-15 09:04:11.698-0500 I have dual-stack here so I cannot be sure if it's isolated to IPv4 or IPv6. I don't have another backtrace at the moment, but I will attach one when I get one. I've enabled _pjsip history_ logging again so it probably won't take long. Hopefully I notice before my PBX is out of service for too long. :-/ By: George Joseph (gjoseph) 2019-05-15 10:23:34.643-0500 OK. When you get another segfault, run... {{/var/lib/asterisk/scripts/ast_coredumper --tarball-coredumps --no-default-search <path_to coredump>}} This will create a tarball of the coredump itself plus the asterisk binaries. We can then use this to pionpoint exactly how the segfault happened. The tarball will be a few hundred megabytes so you may want to host it on dropbox, giigle drive, etc and paste a link in this issue. By: Brian J. Murrell (brian_j_murrell) 2019-05-15 10:42:11.364-0500 Sorry, I'm not sending the coredump itself. As it was, I had to redact some personal/private information out of the stacktrace I provided here. A coredump just has too much potential to leak private information that I do not (and should not) have permission to share. I am sure you understand. I hope so at least. I am otherwise happy to mine any info you want from the coredump. By: George Joseph (gjoseph) 2019-05-15 10:43:51.176-0500 You can send the link to me via email (gjoseph@digium.com) instead of posting it. If not then I totally understand. By: George Joseph (gjoseph) 2019-05-15 10:50:38.197-0500 Oh, if you're good with gdb, then executing a {{p *entry}} might help. The line of code that is mentioned in the backtrace is an "if" statement but the only reference is to "entry". By: Brian J. Murrell (brian_j_murrell) 2019-05-15 11:24:09.862-0500 {noformat} (gdb) p *entry $1 = {number = 101242, transmitted = 0, timestamp = {tv_sec = 1557828826, tv_usec = 0}, src = {sin_family = 10, sin_port = 50195, sin_addr = {s_addr = 0}, sin_zero = " \001A\320\a\000\a\211"}, dst = {sin_family = 0, sin_port = 0, sin_addr = {s_addr = 538968064}, sin_zero = "\000\000\000\000\000\035", <incomplete sequence \370>}, pool = 0x200000000000000, msg = 0x7f9e00000000} {noformat} By: George Joseph (gjoseph) 2019-05-15 11:50:09.737-0500 yeah msg looks suspect. How about {{p *entry->msg}}? By: Brian J. Murrell (brian_j_murrell) 2019-05-15 11:55:41.780-0500 Yeah, no good: {noformat} (gdb) p *entry->msg Cannot access memory at address 0x7f9e00000000 {noformat} By: George Joseph (gjoseph) 2019-05-15 12:12:12.640-0500 I was able to reproduce this by flooding a system with ipv6 requests. By: Brian J. Murrell (brian_j_murrell) 2019-05-15 12:17:03.490-0500 Awesome! By: Brian J. Murrell (brian_j_murrell) 2019-05-31 05:31:03.910-0500 Pity this missed 13.27. Will it be included in a future release? Any idea which one? By: Joshua C. Colp (jcolp) 2019-05-31 05:56:38.228-0500 There is noone actively working on this issue. When a fix is made it'll be put up for review and included, and then go into a subsequent release. When that will be there is no date or guarantee. If you want this fixed sooner you can work on it yourself and provide a patch, you can setup a bug bounty[1], or you can try to elicit interest from someone to do it for free. [1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Bug+Bounties By: Bernhard Schmidt (bschmidt) 2020-08-28 11:31:17.428-0500 This seems to be a duplicate of ASTERISK-28854 By: Joshua C. Colp (jcolp) 2020-08-28 11:34:07.943-0500 Closed as such, thanks! | ||||