You can do this with a join
to the same table but only for the 'Baseline'
. Then, the aggregation only aggregates the values that match, so you should get different baseline averages for the three groups (because the populations are different):
select p.visit, avg(p.weight) as avg_weight, avg(pbl.weight) as avg_blweight,
(avg(p.weight) - avg(pbl.weight)) as change
from phys p join
phys pbl
on p.patient = pbl.patient and
pbl.visit = 'Baseline'
group by p.visit;