How to deal with legacy app storing Arabian characters in latin1 database

Question

I now work on a web-base PHP app to work with a MySQL Server database .
database is with Latin1 Character set and all Persian texts don't show properly .
database is used with a windows software Show and Save Persian texts good .
I don't want to change the charset because windows software work with that charset .

Question:
how can convert latin1 to utf8 to show and utf8 to latin1 for saving from my web-base PHP app , or using Persian/Arabic language on a latin1 charset database without problem ?

note:
one of my texts is احمد رحمانی when save from my windows-based software save as ÇÍãÏ ÑÍãÇäí and still show with احمد رحمانی in my old windows-based software

image : image of database , charsets,collation and windows-based software (full Size)

Answer 1

Edit: Your screenshot shows that the diagnosis below is probably correct.

What to do: Try using iconv() in your PHP web application. You would have to guess or find out what collation/codepage your Windows app uses.

Then something like this might work:

$string_decoded =  iconv("windows-1256", "utf-8", $string); 

You may need to experiment to get this working. Also, I think you need to force your database connection to use latin1 instead of UTF-8!

If you ask me, this is not a good basis for your web app. You would have to convert data into a broken format all the time. You may have to break compatibility with your application, or write an import tool.

The latin1 character set does not cover Persian characters. Proof at collation-charts.org

The only explanation I have why your Delphi program is able to store Arabic characters in a latin1 database is, it could be misusing the latin1 database to store data that isn't covered by latin1, e.g. Windows-1256 Arabic. So the program would store the raw bytes of each arabic character, while in fact these bytes are occupied by other, latin characters in the latin1 character set. But as long as was only the Delphi program storing and fetching the data, no one noticed.

If I'm correct in that - it's the only way I can see how what you describe could be happening - that works as long as only applications are involved that do this the same way, a way which is wrong really.

You should be able to confirm whether this is the case by looking at the data from a "neutral" database tool like phpMyAdmin or HeidiSQL. If you see garbage there instead of Arabic / Persian characters, I may be right.

As to what to do to make your PHP web app work with the same database as your Delphi app - I'm not really sure what to do to be honest. As far as I know, there is no way to force mySQL to use one encoding instead of the other. You would have to manually "re-encode" the data before fetching it into your web app. This is likely to be a painful process.

But first, try to find out what exactly is happening.