[Home]

Summary:ASTERISK-24258: Segmentation fault in ast_variable_update when using app_voicemail.
Reporter:Steven Wheeler (swheeler)Labels:
Date Opened:2014-08-21 16:01:45Date Closed:2017-12-18 11:08:36.000-0600
Priority:MajorRegression?
Status:Closed/CompleteComponents:Applications/app_voicemail Core/General
Versions:1.8.21.0 Frequency of
Occurrence		One Time
Related
Issues:
Environment:CentOS 6.4Attachments:
Description:One of our 1.8.21 servers experienced a segmentation fault while a user was checking their voicemail using app_voicemail. I believe that the segfault happened in {{ast_variable_update}} because the {{category}} variable was null.

In the GDB output you can see that the {{msg_cat}} variable is null. Which is then passed into {{ast_variable_update}}. I don't know enough about the Asterisk source to know if that is a valid value, perhaps a call to {{ast_category_exist}} should happen first? Or maybe we should check for a null value?

GDB Output:
{noformat}
(gdb) bt full
#0  ast_variable_update (category=0x0, variable=0x7fc585c74fce "duration", value=0x7fc57f700980 "19", match=0x0, object=0) at config.c:911
       cur = <value optimized out>
       prev = 0x0
       newer = 0x0
#1  0x00007fc585c6ef4d in vm_forwardoptions (chan=0x7fc574d66788, context=0x7fc57f700c50 "/var/spool/asterisk/voicemail/company-VOICEMAIL/101/INBOX", vms=0x7fc57f700b60, sender=0x7fc57f707c80,
   is_new_message=2138020224, record_gain=0 '\000', urgent=0, fmt=0x7fc585e7f300 "wav49|gsm|wav") at app_voicemail.c:6981
       msg_cat = 0x0
       duration_buf = "19\000\177\305\177\000\000\215\307q", <incomplete sequence \315>
       msgfile = "/var/spool/asterisk/voicemail/company-VOICEMAIL/101/INBOX/msg0000", '\000' <repeats 3013 times>"\220, \274o\177\305\177\000\000\200\274o\177\305\177\000\000\000\000\000\000\000\000\000\000@\276o\177\305\177\000\000\000OY\000\000\000\000\000'\244o\315\305\177\000\000\000OY", '\000' <repeats 13 times>"\205, [\005p\304\177", '\000' <repeats 42 times>, "p\273o\177\305\177\000\000\000\000\000\000\000\000\000\000\376NY", '\000' <repeats 29 times>, "ܻo\177\305\177\000\000\370\273o\177\305\177\000\000\000\000\000\000\000\000\000\000\030\274o\177\305\177", '\000' <repeats 58 times>...
       cmd = 116
       retries = 0
       prepend_duration = 19
       backup_textfile = "/var/spool/asterisk/voicemail/company-VOICEMAIL/101/INBOX/msg0000-bak.txt", '\000' <repeats 4012 times>
       textfile = "/var/spool/asterisk/voicemail/company-VOICEMAIL/101/INBOX/msg0000.txt\000@zo\177\305\177\000\000\000\000\000\000\000\000\000\000 \001\000\000\000\000\000\000 \001\000\000\000\000\000\000P}o\177\305\177", '\000' <repeats 18 times>"\224, \002Y\000\000\000\000\000\030\000\000\000\060\000\000\000 ~o\177\305\177\000\000`}o\177\305\177\000\000`{o\177\305\177\000\000\240zo\177\305\177", '\000' <repeats 18 times>"\200, \002p\177\305\177\000\000\003\000\000\000\000\000\000\000\200\177o\177\305\177\000\000\000\000\000\000\000\000\000\000"...
       msg_cfg = 0x7fc4700396b0
       zero_gain = 0 '\000'
       vm_fmts = 0x7fc585e7f300 "wav49|gsm|wav"
       backup = "/var/spool/asterisk/voicemail/company-VOICEMAIL/101/INBOX/msg0000-bak", '\000' <repeats 4016 times>
       config_flags = {flags = 4}
       duration_str = <value optimized out>
       already_recorded = 1
...
{noformat}

From app_voicemail.c
{noformat}
  6881 static int vm_forwardoptions(struct ast_channel *chan, struct ast_vm_user *vmu, char *curdir, int curmsg, char *vm_fmts,
  6882                         char *context, signed char record_gain, long *duration, struct vm_state *vms, char *flag)
  6883 {
...
  6973                         if (prepend_duration) {
  6974                                 struct ast_category *msg_cat;
  6975                                 /* need enough space for a maximum-length message duration */
  6976                                 char duration_buf[12];
  6977
  6978                                 *duration += prepend_duration;
  6979                                 msg_cat = ast_category_get(msg_cfg, "message");
  6980                                 snprintf(duration_buf, 11, "%ld", *duration);
  6981                                 if (!ast_variable_update(msg_cat, "duration", duration_buf, NULL, 0)) {
  6982                                         ast_config_text_file_save(textfile, msg_cfg, "app_voicemail");
  6983                                 }
  6984                         }
{noformat}

From config.c
{noformat}
   906 int ast_variable_update(struct ast_category *category, const char *variable,
   907                                                 const char *value, const char *match, unsigned int object)
   908 {
   909         struct ast_variable *cur, *prev=NULL, *newer=NULL;
   910
   911         for (cur = category->root; cur; prev = cur, cur = cur->next) {
{noformat}
Comments:By: Matt Jordan (mjordan) 2014-08-22 08:40:43.530-0500

Yes, this should probably check for NULL:

{code}
msg_cat = ast_category_get(msg_cfg, "message");
snprintf(duration_buf, 11, "%ld", *duration);
if (!ast_variable_update(msg_cat, "duration", duration_buf, NULL, 0)) {
ast_config_text_file_save(textfile, msg_cfg, "app_voicemail");
}
{code}

At the same time, you should never *not* have a category in the message envelope file. If that occurred, one of two things has happened:
# Something external manipulated the message envelope file and broke the snot out of it
# Something insidious happened somewhere in Asterisk where it failed to write the message envelope file properly

Can you attach the message envelope file that caused the crash? It should be {{/var/spool/asterisk/voicemail/company-VOICEMAIL/101/INBOX/msg0000.txt}}.
By: Steven Wheeler (swheeler) 2014-08-22 09:16:49.107-0500

We use ODBC storage so there isn't an actual msg000.txt. However, I did find {{/var/spool/asterisk/voicemail/company-VOICEMAIL/101/INBOX/msg0000-bak.txt}} which is referenced in the GDB output.  Unfortunately it was an empty file. Would the category go by a different name in the ODBC storage table?
By: Matt Jordan (mjordan) 2014-08-22 11:34:00.925-0500

Nope, as that's not what {{msg_cfg}} was opened as.

{code}
if ((msg_cfg = ast_config_load(textfile, config_flags)) && valid_config(msg_cfg) && (duration_str = ast_variable_retrieve(msg_cfg, "message", "duration"))) {
*duration = atoi(duration_str);
} else {
*duration = 0;
}
{code}

The odd thing is that prior to accessing the file, we do check to see if it is valid:

{code}
/* if we can't read the message metadata, stop now */
if (!valid_config(msg_cfg)) {
cmd = 0;
break;
}
{code}

The backup text file is the original text file prior to the pre-pending of data. It looks like the pre-pending message envelope file should be created as part of {{vm_forwardoptions}}, but apparently something went wrong somewhere between the check for validity and where we actually save the {{msg_cfg}} back out to the {{textfile}}.

I'd say that adding a check that {{ast_cagegory_get}} retrieves a context would be the right thing to do - and if it doesn't get it, bail out. The other option would be to try and recover the envelope information if the {{message}} context doesn't exist.

It would be nice to know if this crash is reproducible, or if this was a one-off occurrence. If it is reproducible, that would help narrow down how the message envelope file is empty.
By: Steven Wheeler (swheeler) 2014-08-22 11:42:20.301-0500

This is the only time we have seen Asterisk crash like this. I haven't been able to reproduce it yet. Checking for null and then handling it gracefully seems like the right solution.
By: Joshua C. Colp (jcolp) 2017-12-18 09:40:12.119-0600

Did this issue ever reoccur for you since your last comment?
By: Steven Wheeler (swheeler) 2017-12-18 09:45:25.733-0600

No, we haven't seen a crash like this since then.
By: Joshua C. Colp (jcolp) 2017-12-18 11:08:36.095-0600

I'm suspending this then since it was a one time problem on an old now unsupported version of Asterisk. If it occurs again you can comment and this will automatically reopen.