Summary: | ASTERISK-25489: Crash while calling function iax2_frame_free | ||
Reporter: | Y Ateya (yateya) | Labels: | |
Date Opened: | 2015-10-22 16:56:16 | Date Closed: | 2015-11-04 18:54:52.000-0600 |
Priority: | Major | Regression? | |
Status: | Closed/Complete | Components: | Channels/chan_iax2 |
Versions: | 13.4.0 | Frequency of Occurrence | Occasional |
Related Issues: | |||
Environment: | Ubuntu 14.4, x86 machine built from source. | Attachments: | |
Description: | Periodically (every 1-3 days) I have the same crash on my asterisk server.
{noformat} #0 0x00002b8899790cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00002b88997940d8 in __GI_abort () at abort.c:89 #2 0x00002b88997cd394 in __libc_message (do_abort=do_abort@entry=1, fmt=fmt@entry=0x2b88998dbb28 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175 #3 0x00002b88997d966e in malloc_printerr (ptr=<optimised out>, str=0x2b88998dbc58 "double free or corruption (out)", action=1) at malloc.c:4996 #4 _int_free (av=<optimised out>, p=<optimised out>, have_lock=0) at malloc.c:3840 #5 0x00002b8910a21ad9 in iax2_frame_free (fr=0x2b89b084d0e0) at chan_iax2.c:2149 #6 __attempt_transmit (data=0x2b89b084d0e0) at chan_iax2.c:3647 #7 0x00002b8910a3fadd in iax2_process_thread (data=data@entry=0x1a06d10) at chan_iax2.c:12534 #8 0x00000000005db08a in dummy_start (data=<optimised out>) at utils.c:1237 #9 0x00002b8898df3182 in start_thread (arg=0x2b8918a11700) at pthread_create.c:312 #10 0x00002b889985447d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 {noformat} Just before this crash, the messages log shows these errors couple of times: {noformat} [2015-10-20 08:05:13.292] NOTICE[24401][C-000b5c9c] res_rtp_asterisk.c: Unknown RTP codec 48 received from '12.22.16.91:22822' [2015-10-20 08:05:13.340] WARNING[24401][C-000b5c9c] res_rtp_asterisk.c: RTP Read too short (32, expecting 48 [2015-10-20 08:05:13.410] WARNING[24401][C-000b5c9c] res_rtp_asterisk.c: RTP Read too short (32, expecting 48 [2015-10-20 08:05:13.484] WARNING[24401][C-000b5c9c] res_rtp_asterisk.c: RTP Read too short (42, expecting 48 [2015-10-20 08:05:13.568] NOTICE[24401][C-000b5c9c] res_rtp_asterisk.c: Unknown RTP codec 48 received from '12.22.16.91:22822' [2015-10-20 08:05:13.650] NOTICE[24401][C-000b5c9c] res_rtp_asterisk.c: Unknown RTP codec 48 received from '12.22.16.91:22822' [2015-10-20 08:05:13.652] ERROR[24410][C-000b5ca4] chan_iax2.c: Unable to schedule iax2 callno 23820 destruction?!! Destroying immediately. [2015-10-20 08:05:13.722] WARNING[24401][C-000b5c9c] res_rtp_asterisk.c: RTP Read too short (42, expecting 48 [2015-10-20 08:05:13.804] NOTICE[24401][C-000b5c9c] res_rtp_asterisk.c: Unknown RTP codec 48 received from '12.22.16.91:22822' [2015-10-20 08:05:13.894] NOTICE[24401][C-000b5c9c] res_rtp_asterisk.c: Unknown RTP codec 48 received from '12.22.16.91:22822' [2015-10-20 08:05:13.954] ERROR[24359][C-000b5c75] chan_iax2.c: Unable to schedule iax2 callno 20795 destruction?!! Destroying immediately. [2015-10-20 08:05:13.980] NOTICE[24401][C-000b5c9c] res_rtp_asterisk.c: Unknown RTP codec 48 received from '12.22.16.91:22822' [2015-10-20 08:05:14.057] NOTICE[24401][C-000b5c9c] res_rtp_asterisk.c: Unknown RTP codec 48 received from '12.22.16.91:22822' [2015-10-20 08:05:14.536] NOTICE[24401][C-000b5c9c] res_rtp_asterisk.c: Unknown RTP codec 48 received from '12.22.16.91:22822' [2015-10-20 08:05:14.548] ERROR[24502][C-000b5cde] chan_iax2.c: Unable to schedule iax2 callno 20140 destruction?!! Destroying immediately. [2015-10-20 08:05:14.608] NOTICE[24401][C-000b5c9c] res_rtp_asterisk.c: Unknown RTP codec 48 received from '12.22.16.91:22822' [2015-10-20 08:05:14.919] ERROR[24372][C-000b5c82] chan_iax2.c: Unable to schedule iax2 callno 29778 destruction?!! Destroying immediately. [2015-10-20 08:05:14.922] ERROR[22813][C-000b57dd] chan_iax2.c: Unable to schedule iax2 callno 28017 destruction?!! Destroying immediately. [2015-10-20 08:05:15.293] ERROR[24351][C-000b5c70] chan_iax2.c: Unable to schedule iax2 callno 25656 destruction?!! Destroying immediately. [2015-10-20 08:05:15.319] ERROR[24266][C-000b5c14] chan_iax2.c: Unable to schedule iax2 callno 30734 destruction?!! Destroying immediately. {noformat} Probably it is caused by this code segment in `channels/iax2/parser.c` {code} if (!fr->cacheable || !(iax_frames = ast_threadstorage_get(&frame_cache, sizeof(*iax_frames)))) { ast_free(fr); return; } {code} At the time of crash the system has a lot of memory (not out of memory), cpu usage was normal. I tried to re-produce the bug on test server but I failed. It only happens on production servers. | ||
Comments: | By: Asterisk Team (asteriskteam) 2015-10-22 16:56:17.222-0500 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. By: Rusty Newton (rnewton) 2015-10-30 16:42:25.546-0500 Thank you for the crash report. However, we need more information to investigate the crash. Please provide: 1. A backtrace generated from a core dump using the instructions provided on the Asterisk wiki [1]. 2. Specific steps taken that lead to the crash. 3. All configuration information necesary to reproduce the crash. Thanks! [1]: https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace Please supply the backtrace as described on the wiki. Remember to recompile with the appropriate compiler flags. By: Rusty Newton (rnewton) 2015-10-30 16:43:04.415-0500 If you can - please gather a debug log along with the crash trace. https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information By: Y Ateya (yateya) 2015-11-02 16:48:37.262-0600 This issue was solved by this commit https://gerrit.asterisk.org/#/c/1254/ . The main cause was wrap around of scheduler ID. Upgrading to 13.6 solves the problem. |