utf8::is_utf8
does not tell you whether your string is encoded using UTF-8 or not. (That information is not even available.)
>perl -MEncode -E"say utf8::is_utf8(encode_utf8(chr(0xE9))) || 0"
0
utf8::valid
does not tell you whether your string is valid UTF-8 or not.
>perl -MEncode -E"say utf8::valid(qq{\xE9}) || 0"
1
Both check some internal storage details. You should never have a need for either.
File::Queue can only transmit strings of bytes. It's up to you to serialise the data you want to transmit into a string.
The primary means of serialising text is character encoding, or just encoding for short. UTF-8 is a character encoding.
For example, the string
dostępu
consists of the following chars (each a Unicode code point):
64 6F 73 74 119 70 75
Not all of those chars fit in bytes, so the string can't be sent using File::Queue. If you were to encode that string using UTF-8, you'd get a string composed of the following chars:
64 6F 73 74 C4 99 70 75
Those chars fit in bytes, so that string can be sent using File::Queue.
JSON, as you used it, returns strings of Unicode code points. As such, you need to apply a character encoding.
File::Queue doesn't provide an option to automatically encode strings for you, so you'll have to do it yourself.
You could use encode_utf8
and decode_utf8
from the Encode module
my $json = JSON->new->allow_nonref;
$q->enq(encode_utf8($json->encode(\%data)));
my $data = $json->decode(decode_utf8($q->deq()));
or you can let JSON do the encoding/decoding for you.
my $json = JSON->new->utf8->allow_nonref;
$q->enq($json->encode(\%data));
my $data = $json->decode($q->deq());