I think the first approach will be:
select MAX(a) as a,
b
from t as t1
where NOT EXISTS(select a from t where b<>t1.b and a=t1.a)
or
NOT EXISTS(select a from t where a in (select a from t where b=t1.b)
GROUP BY a
HAVING COUNT(*)=1)
GROUP BY b
We should group by b
and find MAX(a)
but from the special subrange of the main table.
First we should get all a
which aren't exist for another b
(it is the first condition).
But in the case of b=3
we get the case that ALL a
exist for other b
so the second condition handles this case.