Clickatell arabic text getting garbled

Question

I ma trying to send arabic text from the clickatell sms provider using php, but all i have been able to send yet is garbled text. The text I am using for testing is "غثس هفس ".

I have tried encoding the message text to Windows-1256,1250 and ISO-8859-6 formats using iconv but everytimeit just sends garbled texts.

Can anyone please give me some pointers to what I am missing?

Thanks in advance

Answer 1

Ok, this is for whoever comes on this path later. Found the solution here Retaining code from Esailija's answer

<?php
$text = "غثس هفس خن";
$arr = unpack('H*hex', iconv('UTF-8', 'UCS-2BE', $text));
$message = strtoupper($arr['hex']);

$username = '';
$password = '';
$API_ID = '';
$to = '';

$url = "http://api.clickatell.com/http/auth?user=$username&password=$password&api_id=$API_ID";
$ret = file($url);
$sess = explode(":",$ret[0]);
if ($sess[0] == "OK") {
    $sess_id = trim($sess[1]);
    $url = "http://api.clickatell.com/http/sendmsg?session_id=$sess_id&to=$to&text=$message&unicode=1";
    $ret = file($url);
    $send = explode(":",$ret[0]);

    if ($send[0] == "ID") {
        echo "success - message ID: ". $send[1];
    } else {
        echo "send message failed";
    }
} else {
    echo "Authentication failure: ". $ret[0];
}

Answer 2

Short answer: you need to do two things:

a) Convert your Arabic message:

$data = iconv("UTF-8", "UCS-2BE", $data); $data = bin2hex($data);

Use the resulting string as your message text

b) When sending via the HTTP API, include the unicode parameter:

&unicode=1

Answer 3

Based on this, the default is GSM but you can also choose UCS2.

So:

<?php
                                 //Make sure the PHP source file is physically
                                //saved in utf-8
$text = rawurlencode(iconv( "UTF-8", "UCS-2", "غثس هفس "));

$url = "http://api.clickatell.com/http/auth?user=username&password=password&api_id=API_ID&encoding=UCS2";
$ret = file($url);
$sess = explode(":",$ret[0]);
if ($sess[0] == "OK") {

    $sess_id = trim($sess[1]);
    $url = "http://api.clickatell.com/http/sendmsg?session_id=$sess_id&to=$to&text=$text&encoding=UCS2";
    $ret = file($url);
    $send = explode(":",$ret[0]);

    if ($send[0] == "ID") {
        echo "successnmessage ID: ". $send[1];
    } else {
        echo "send message failed";
    }
} else {
    echo "Authentication failure: ". $ret[0];
}

Answer 4

I rewrote ClickATell code for Vtiger 6 so that it can accommodate UTF-8 Extended characters. I use Turkish. I use the below code for conversion. I hope you can port.

/**
 * Function to handle UTF-8 Check and conversion
 * @author Nuri Unver
 * 
 */
public function smstxtcode($data){ 
$mb_hex = '';
$utf = 0;
for($i = 0 ; $i<mb_strlen($data,'UTF-8') ; $i++){ 
    $c = mb_substr($data,$i,1,'UTF-8');

    $o = unpack('N',mb_convert_encoding($c,'UCS-4BE','UTF-8')); 
    $hx = sprintf('%04X',$o[1]); 
    $utf += intval(substr($hx,0,2));
    $mb_hex .= $hx;
} 
if ($utf>0)
{
    $return=$mb_hex;
    $utf=1;
}
else
{
    $return=utf8_decode($data);
    $utf=0;
}
return array($utf,$return); 

}

The you call this function with your message. The response you get it is an array saying wheter to send unicode or normal text depending on the message and also the text to be sent. If there are no extended characters, it just sends it as plain text with unicode=0 in order to save characters. If message contains extended characters, it converts the message to hexcode and sends it as unicode.

This code just does the calculations. You need to implement your own code to port it to your system. For demonstration this is the code I use for Vtiger to extract data and send the message:

/**
 * Function to handle SMS Send operation
 * @param <String> $message
 * @param <Mixed> $toNumbers One or Array of numbers
 */
public function send($message, $toNumbers) {
    if(!is_array($toNumbers)) {
        $toNumbers = array($toNumbers);
    }
    $params = $this->prepareParameters();
    $smsarray = $this->smstxtcode($message);
    $params['text'] = $smsarray[1];
    $params['unicode'] = $smsarray[0];
    $params['to'] = implode(',', $toNumbers);
    $serviceURL = $this->getServiceURL(self::SERVICE_SEND);