How to get correct list position in multi-byte string using preg_match

Question

I am currently matching HTML using this code:

preg_match('/<\/?([a-z]+)[^>]*>|&#?[a-zA-Z0-9]+;/u', $html, $match, PREG_OFFSET_CAPTURE, $position)

It matches everything perfect, however if I have a multibyte character, it counts it as 2 characters when giving back the position.

For example the returned $match array would give something like:

array
  0 => 
    array
      0 => string '<br />' (length=6)
      1 => int 132
  1 => 
    array
      0 => string 'br' (length=2)
      1 => int 133

The real number for the <br /> match is 128, but there are 4 multibyte characters, so it's giving 132. I really thought adding the /u modifier would make it realize what's going on, but no luck there.

Answer 1

I looked at this suggestion from @Qtax:

UTF-8 characters in preg_match_all (PHP)

And for some more reference, this bug surfaced while using this: Truncate text containing HTML, ignoring tags

The gist of the change is this:

$orig_utf = 'UTF-8';
$new_utf  = 'UTF-32';

mb_regex_encoding( $new_utf );

$html     = mb_convert_encoding( $html, $new_utf, $orig_utf );
$end_char = mb_convert_encoding( $end_char, $new_utf, $orig_utf );


mb_ereg_search_init( $html );

$pattern = '</?([a-z]+)[^>]*>|&#?[a-zA-Z0-9]+;';
$pattern = mb_convert_encoding( $pattern, $new_utf, $orig_utf );

while ( $printed < $limit && $tag_match = mb_ereg_search_pos( $pattern, $html ) ) {

  $tag_position = $tag_match[0]/4;
  $tag_length   = $tag_match[1];
  $tag          = mb_substr( $html, $tag_position, $tag_length/4, $new_utf );
  $tag_name     = preg_replace( '/[\s<>\/]+/', '', $tag );

  // Print text leading up to the tag.
  $str = mb_substr($html, $position, $tag_position - $position, $new_utf );

  .......

} 

Also in reference to the truncate HTML page, there are other neccessary changes:

$first_char = mb_substr( $tag, 0, 1, $new_utf );

if ( $first_char == mb_convert_encoding( '&', $new_utf ) ) {
  ...
}

My text editor is UTF-8 so if I was comparing the 32 to my file's ampersand, it wouldn't work.

Answer 2

If you need quick fix and don't care about speed:

$mb_pos = mb_strlen( substr($string, 0, $pos) );

Answer 3

Have you looked into http://www.php.net/manual/en/function.mb-ereg.php ?