You don't want to loop over all the $values
in the inner loop, you just want to explode the one with the same index as the current attribute and loop over those.
foreach ($attributes as $index => $v) {
echo "<b>{$v['attribute']}</b><br>";
foreach (explode(',', $values[$index]['value']) as $explval) {
$qryString = array( 'search' => $search,
'attr' => $explval
);
echo anchor('products/item_search?'. http_build_query($qryString), $explval) ."<br>";
}
}