You listed three measurements:
- Average (mean) response
- Median response
- 5-95 span response
Notice that #3 is not measuring the same thing as #1 and #2!
- Mean and median give you a measure of the actual response time. This will pick up a certain class of problem.
- 5-95 span tells you to what extent your response time varies. i.e. Is your response time consistent or not. This will pick up another class of problem.
You probably need to track both: the absolute response time, as well as the variance. The best approach for the former (mean vs median, whether to clip outliers) probably depends on the results you get for your service.