Here you go:
SELECT prod_id, (((msrp + 0.0) - (
CASE
WHEN v.charges_by = 'Order'
THEN p.cost + v.fee_per_order + v.extra_fee_per_product
WHEN v.charges_by = 'Product'
THEN p.cost + p.fee_per_product + v.extra_fee_per_product
END
)) / msrp) * 100 AS Margin
FROM Vendors v
INNER JOIN Products p
ON v.id = p.vendor_id
HAVING Margin BETWEEN 10 AND 100
Working Fiddle: http://sqlfiddle.com/#!2/ddbaf2/8
The critical part here is msrp + 0.0
, it has been done to treat the calculation as a floating point value instead of a number.