I think you can do this with window functions:
select
*
from (
select
user_id,
survey_id,
row_number() over (partition by user_id order by survey_id desc) rn
from
all_survey_res
) x
where
x.rn <= 4