If the question is "Can I do it without the colon and empty quotes?" The answer is no you cannot. You must have the closing :''
and it is best to use paren's to clarify your desires.
$var = 'here is the first part and '.
(( $foo == $bar ) ? "the optional middle part":'') .
' and the rest of the string.'
I think the biggest problem here is that you're trying to do things inline. This basically boils down to the same process and does not use an unclosed ternary:
$var = 'here is the first part and ';
if( $foo == $bar ) $var .= "the optional middle part";
$var .= ' and the rest of the string.';
Whereas this is another way to accomplish the same goal without needing to worry about conditionals breaking the string:
$middle = '';
if( $foo == $bar ) $middle = ' the optional middle part and';
$var = sprintf('here is the first part and%s the rest of the string.',$middle);
Now, if you are going to be needlessly clever, I suppose you could do this instead:
$arr = array('here is the first part and',
'', // array filter will remove this part
'here is the end');
// TRUE evaluates to the key 1.
$arr[$foo == $bar] = 'here is the middle and';
$var = implode(' ', array_filter($arr));