You should be able to use the same query as your first one but add a filter to exclude any rooms that have users outside of 2, 5:
select roomId
from user_to_room
where userid in (2,5)
and roomid not in (select roomid
from user_to_room
where userid not in (2, 5))
group by roomId
having count(distinct userid) = 2;