[Home]

Summary:ASTERISK-28830: Incorrect UTF-8 handling when using function FILTER
Reporter:newborn (newborn838)Labels:
Date Opened:2020-04-14 10:40:43Date Closed:
Priority:MajorRegression?
Status:Open/NewComponents:Functions/General
Versions:13.31.0 Frequency of
Occurrence		Constant
Related
Issues:
Environment:Attachments:
Description:As you can see, after using FILTER we're getting wrong symbols (Коп▒▒а")
This causes dialplan errors and crash
How to filter UTF-8 (Russian alphabet) correctly?

Code
{noformat}
NoOp(.. Processing call from ${CALLERID(all)} to ${EXTEN}@${CONTEXT}-GLOBAL ..);
Set(CALLERID(name)=${FILTER(А-Яа-яA-Za-z\040\050\051\055\056,${CALLERID(name)})});
{noformat}
Result
{noformat}
[Apr 14 18:26:08]     -- Executing [0999@inbound-trunk:1] NoOp("IAX2/02asterisk03-2864", ".. Processing call from "ООО Рога и Копыта" <7***> to 0299@inbound-trunk-GLOBAL ..") in new stack
[Apr 14 18:26:08]     -- Executing [0299@inbound-trunk:2] Set("IAX2/msk02asterisk03-11656", "CALLERID(name)=ООО Рога и Коп▒▒а") in new stack
[Apr 14 18:26:08] WARNING[1322]: res_odbc.c:550 ast_odbc_print_errors: SQL Execute returned an error: 22021: ERROR: invalid byte sequence for encoding "UTF8": 0xd1 0xd1;
Error while preparing parameters
[Apr 14 18:26:08] WARNING[1322]: res_odbc.c:445 ast_odbc_prepare_and_execute: SQL Execute error -1!
[Apr 14 18:26:08] WARNING[1322]: cel_odbc.c:797 odbc_log: Insert failed on 'asterisk-local:cel'.  CEL failed: INSERT INTO cel

[Apr 14 18:26:16]     -- Executing [s@macro-inboundcall:5] Set("SIP/00001a28", "CALLFILENAME=in-2020-04-14-Tuesday--18h26m--from--ООО Рога и Коп▒▒а-7***--to-999--1586877968.66779") in new stack
[Apr 14 18:26:16] ERROR[21790][C-00000df1]: json.c:870 ast_json_vpack: Error building JSON from '{s: s, s: s}': Invalid UTF-8 string.
[Apr 14 18:26:16] ERROR[21790][C-00000df1]:   Got 19 backtrace records
# 0: /usr/sbin/asterisk(ast_json_pack+0x99) [0x4ff6e9]
# 1: /usr/sbin/asterisk(ast_channel_publish_varset+0x25) [0x5874a5]
# 2: /usr/sbin/asterisk(pbx_builtin_setvar_helper+0x123) [0x54b143]
# 3: /usr/sbin/asterisk(pbx_builtin_setvar+0xc6) [0x54b3f6]
# 4: /usr/sbin/asterisk(pbx_exec+0xb9) [0x5432d9]
# 5: /usr/sbin/asterisk() [0x53b92e]
# 6: /usr/sbin/asterisk(ast_spawn_extension+0x30) [0x53c410]
# 7: /usr/lib64/asterisk/modules/app_macro.so(+0x2d17) [0x7f89f309dd17]
# 8: /usr/sbin/asterisk(pbx_exec+0xb9) [0x5432d9]
# 9: /usr/sbin/asterisk(ast_app_exec_macro+0x5e) [0x443f6e]
#10: /usr/lib64/asterisk/modules/app_dial.so(+0xbd0d) [0x7f89f6118d0d]
#11: /usr/lib64/asterisk/modules/app_dial.so(+0xd2d6) [0x7f89f611a2d6]
#12: /usr/sbin/asterisk(pbx_exec+0xb9) [0x5432d9]
#13: /usr/sbin/asterisk() [0x53b92e]
#14: /usr/sbin/asterisk() [0x53c95e]
#15: /usr/sbin/asterisk() [0x53de1b]
#16: /usr/sbin/asterisk() [0x5a3f4e]
#17: /lib64/libpthread.so.0(+0x7e65) [0x7f8a2ec4fe65]
#18: /lib64/libc.so.6(clone+0x6d) [0x7f8a2e22088d]
{noformat}
Comments:By: Asterisk Team (asteriskteam) 2020-04-14 10:40:43.966-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.
By: George Joseph (gjoseph) 2020-04-14 13:41:17.926-0500

We appreciate the difficulties you are facing, however this does not appear to be a bug report and your request or comments would be better served in a different forum.

The Asterisk community provides support over IRC, mailing lists, and forums [2] as described at http://asterisk.org/community. The Asterisk issue tracker is used specifically to track issues concerning bugs and documentation errors.

Please see the Asterisk Issue Guidelines [1] for instruction on the intended use of the Asterisk issue tracker.

Thanks!

[1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines
[2] https://community.asterisk.org


The characters you're trying to filter out {{ыт}} are the values {{\d18b}} and {{\d182}}.  Your filter allows {{а-я}} which is {{\d0b0-\d18f}} so those characters are allowed.  You need to adjust your filter appropriately.

If you have more question about the FILTER function, please visit the community forums linked above.

By: newborn (newborn838) 2020-04-15 08:28:53.774-0500

Have you read the code and the output?
There is no problem with allowed characters. The problem is that FILTER function substitutes some characters while first argument is passed as a range of characters and the second argument passed as a variable (CALLERID in my case) which causes system to crash and error.

$\{FILTER(а-я,$\{CALLERID(name)})} - Crash, some characters in the second argument are broken and substituted with garbage

$\{FILTER(абвгдеёжзийклмнопрстуфхцчшщъыьэюя,$\{CALLERID(name)})} - No crash, no substitutes

By: Asterisk Team (asteriskteam) 2020-04-15 08:28:54.002-0500

This issue has been reopened as a result of your commenting on it as the reporter. It will be triaged once again as applicable.
By: George Joseph (gjoseph) 2020-04-15 08:36:10.494-0500

Sorry about that.  I misunderstood.  I'll take a look again.

By: George Joseph (gjoseph) 2020-04-15 09:50:39.910-0500

The base issue is that the string dialplan functions like FILTER are not UTF aware.
The filter string {{А-Яа-яA-Za-z}}  {{d090 2dd0 afd0 b02d d18f 412d 5a61 2d7a}} is being interpreted as
Allow: {{d0 90-d0 af b0-d1 8f 41-5a 61-7a}}

The 2 characters being altered are {{ыт}} {{d18b d182}} because although {{d1}} is allowed, {{8b}} and {{82}} are not.  When the new string gets down to the SQL layer, those 2 characters appear as {{d1d1}} which is invalid.

I've created an internal issue to make those functions UTF aware but in the mean time the only thing I can suggest is to add another range to your filter string that has the low order bytes for the UTF-8 Cryillic characters.  {{\x80-\xbf}} should do it.  So your filter string would be {{А-Яа-яA-Za-z\x80-\xbf\040\050\051\055\056}}

By: newborn (newborn838) 2020-04-15 10:32:21.969-0500

Ok,
As a temporary solution i have removed А-Яа-я and used explicit alphabet characters like $\{FILTER(абвгдеёжзийклмнопрстуфхцчшщъыьэюя,$\{CALLERID(name)})}
By: Dmitriy (teramosk) 2020-04-22 04:05:30.487-0500

I Have similar problem. It is not problem of FILTER function - it is global problem

Example1
Dialplan
same => n,NoOp("Это не конференция, значит включаем на канале запись разговора и фичю для возможности создания конференции налету")

CLI Result 1:
-- Executing [s@AFTER_ANSWER:14] GotoIf("SIP/AVAYA-00003334", "0?conf:record") in new stack
-- Goto (AFTER_ANSWER,s,18)
-- Executing [s@AFTER_ANSWER:18] NoOp("SIP/AVAYA-00003334", "Это не конференция, значит включаем на канале запись разговора и фичю для возможности создания конфе    -- Executing [s@AFTER_ANSWER:19] ExecIf("SIP/AVAYA-00003334", "0?SET(DYNAMIC_FEATURES=conference_callee)") in new stack
-- Executing [s@AFTER_ANSWER:20] Set("SIP/AVAYA-00003334", "recordDir=2020-04-22 13:03:20") in new stack

CLI Result 2:
-- Executing [s@AFTER_ANSWER:14] GotoIf("SIP/4253-00003353", "0?conf:record") in new stack
-- Goto (AFTER_ANSWER,s,18)
-- Executing [s@AFTER_ANSWER:18] NoOp("SIP/4253-00003353", "Это не конференция, значит включаем на канале запись разговора и фичю для возможности создания конференции налету") in new stack
-- Executing [s@AFTER_ANSWER:19] ExecIf("SIP/4253-00003353", "1?SET(DYNAMIC_FEATURES=conference_callee)") in new stack

CLI Result 3:
-- Executing [s@AFTER_ANSWER:14] GotoIf("SIP/AVAYA-00003356", "0?conf:record") in new stack
-- Goto (AFTER_ANSWER,s,18)
-- Executing [s@AFTER_ANSWER:18] NoOp("SIP/AVAYA-00003356", "Это не конференция, значит включаем на канале запись разговора и фичю для возможности создания конфере▒    -- Executing [s@AFTER_ANSWER:19] ExecIf("SIP/AVAYA-00003356", "0?SET(DYNAMIC_FEATURES=conference_callee)") in new stack
-- Executing [s@AFTER_ANSWER:20] Set("SIP/AVAYA-00003356", "recordDir=2020-04-22 13:07:26") in new stack

Example2
Crash CDR recording
[Apr 16 05:08:57] WARNING[1301]: res_odbc.c:538 ast_odbc_print_errors: SQL Execute returned an error: HY000:
[MySQL][ODBC 8.0(a) Driver][mysqld-5.7.29-0ubuntu0.18.04.1]Incorrect string value: '\xD0' for column 'lastdata' at row 1
[Apr 16 05:08:57] WARNING[1301]: res_odbc.c:433 ast_odbc_prepare_and_execute: SQL Execute error -1!
[Apr 16 05:08:57] WARNING[1301]: cdr_adaptive_odbc.c:765 odbc_log: cdr_adaptive_odbc: Insert failed on 'asteriskcdrdb:cdr'.  
CDR failed: INSERT INTO cdr (calldate, clid, src, dst, dcontext, channel, dstchannel, lastapp, lastdata, duration, billsec, disposition, amaflags, accountcode, uniqueid, recordingfile, cnum, cnam, linkedid, peeraccount, sequence) VALUES ({ ts '2020-04-16 05:06:36' }, '"Оператор 03" <xxxx>', 'xxxx', 'xxxx', 'FROM_PHONE_03', 'SIP/101-00000095', 'SIP/TRUNK_E1_GATE-00000096', 'Dial', 'SIP/TRUNK_E1_GATE/xxxx,,gU(ABONENT_ANSWER,101,Максимовская ▒', 140, 132, 'ANSWERED', 3, '955', '1586995596.156', '2020/04/16/2020-04-16 05:06:36 - 1586995596.157 (101[Максимовская С.С] -> xxxx).mp3', '101', 'Максимовская С.С', '1586995596.156', '955', 179)

Example3
CLI>database show
/agent/0131/name                                  : Демина АВ
/agent/0132/name                                  : Домра▒       > ева АА

I have this bug on other OS and other Asterisk versions
Ubuntu 12.04, 16.04, 18.04
CentOS 8
Asterisk 13.18-cert2
Asterisk 13.21-cert6
Asterisk 16.3-cert6
Asterisk 16.9.0

By: Dmitriy (teramosk) 2020-04-23 01:50:05.798-0500

I found scenarios in which both types of errors are stably reproduced.

; BUG 1. Incorrect UTF8 text output
[FROM_PHONE_03]
exten => 8,1,NoOp()
same => n,Set(n=20)
same => n,While($[${SET(n=$[${n}-1])} > 0])
same => n,NoOp(ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта)
same => n,EndWhile
same => n,Hangup()

RESULT:
 == Using SIP RTP CoS mark 5
   -- Executing [8@FROM_PHONE_03:1] NoOp("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:2] Set("SIP/101-0000008b", "n=20") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРа▒       >        > ▒еромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИП▒       >        > ▒ВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутренн▒       >        > ▒оБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтро▒       >        > ▒СодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДва▒       >        > ▒дельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГра▒       >        > ▒цуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭт▒       >        > ▒уквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистические▒       >        > ▒мволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПять▒       > отДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаК       > ▒нсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфера▒       > стерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержи       > ▒КирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКакоготоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельны       > ▒Байта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "1") in new stack
   -- Executing [8@FROM_PHONE_03:4] NoOp("SIP/101-0000008b", "ЕслиСтрокаСодержитКирилицуИПриВыводеНаКонсольКирилистическиеСимволыПопадаютНаГраницуКак▒       > готоВнутреннегоБуфераАстерискаРазмеромВПятьсотДвенадцатьБайтТоЭтуБуквуРазрываетНаДваОтдельныхБайта") in new stack
   -- Executing [8@FROM_PHONE_03:5] EndWhile("SIP/101-0000008b", "") in new stack
   -- Executing [8@FROM_PHONE_03:3] While("SIP/101-0000008b", "0") in new stack
   -- Executing [8@FROM_PHONE_03:6] Hangup("SIP/101-0000008b", "") in new stack
 == Spawn extension (FROM_PHONE_03, 8, 6) exited non-zero on 'SIP/101-0000008b'


; BUG 2. Incorrect UTF8 write to CDR mysql table using cdr_adaptive_odbc
[FROM_PHONE_03]
exten => 9,1,NoOp()
same => n,Set(CDR(userfield)=АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэ яАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ) ; Success Insert To CDR table
;same => n,Set(CDR(userfield)=АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ) ; Failed  Insert To CDR table
same => n,Answer()
same => n,Hangup()

;stably SUCCESS if I use first variant userfield
 == Using SIP RTP CoS mark 5
   -- Executing [9@FROM_PHONE_03:1] NoOp("SIP/101-0000008c", "") in new stack
   -- Executing [9@FROM_PHONE_03:2] Set("SIP/101-0000008c", "CDR(userfield)=АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэ яАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ") in new stack
   -- Executing [9@FROM_PHONE_03:3] Answer("SIP/101-0000008c", "") in new stack
   -- Executing [9@FROM_PHONE_03:4] Hangup("SIP/101-0000008c", "") in new stack
 == Spawn extension (FROM_PHONE_03, 9, 4) exited non-zero on 'SIP/101-0000008c'


[FROM_PHONE_03]
exten => 9,1,NoOp()
;same => n,Set(CDR(userfield)=АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэ яАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ) ; Success Insert To CDR table
same => n,Set(CDR(userfield)=АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ) ; Failed  Insert To CDR table
same => n,Answer()
same => n,Hangup()

;stably FAILED if I use second variant userfield
 == Using SIP RTP CoS mark 5
   -- Executing [9@FROM_PHONE_03:1] NoOp("SIP/101-0000008d", "") in new stack
   -- Executing [9@FROM_PHONE_03:2] Set("SIP/101-0000008d", "CDR(userfield)=АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ") in new stack
   -- Executing [9@FROM_PHONE_03:3] Answer("SIP/101-0000008d", "") in new stack
   -- Executing [9@FROM_PHONE_03:4] Hangup("SIP/101-0000008d", "") in new stack
 == Spawn extension (FROM_PHONE_03, 9, 4) exited non-zero on 'SIP/101-0000008d'
[Apr 23 11:41:02] WARNING[31190]: res_odbc.c:538 ast_odbc_print_errors: SQL Execute returned an error: HY000: [MySQL][ODBC 8.0(a) Driver][mysqld-5.7.29-0ubuntu0.18.04.1]Incorrect string value: '\xD1' for column 'userfield' at row 1
[Apr 23 11:41:02] WARNING[31190]: res_odbc.c:433 ast_odbc_prepare_and_execute: SQL Execute error -1!
[Apr 23 11:41:02] WARNING[31190]: cdr_adaptive_odbc.c:765 odbc_log: cdr_adaptive_odbc: Insert failed on 'asteriskcdrdb:cdr'.  CDR failed: INSERT INTO cdr (calldate, clid, src, dst, dcontext, channel, lastapp, duration, billsec, disposition, amaflags, uniqueid, userfield, linkedid, sequence) VALUES ({ ts '2020-04-23 11:41:01' }, '"ОператорОператорОператорОператорОперато', '101', '9', 'FROM_PHONE_03', 'SIP/101-0000008d', 'Hangup', 0, 0, 'ANSWERED', 3, '1587624061.371', 'АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэю▒', '1587624061.371', 171)

By: George Joseph (gjoseph) 2020-06-03 10:56:28.687-0500

[~teramosk] Unfortunately, I had to delete the patch attachment because you haven't signed a contributor license agreement.
If you'd like to resubmit the patch, please use the link at the top of the page to sign a license agreement and follow the instructions in the link below to submit the patch to Gerrit.
https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process