Actually you are not allowed transactions inside stored functions. You are allowed transactions inside stored procedures only.
create procedure test( a int )
MODIFIES SQL DATA
BEGIN
START TRANSACTION ;
update t set col='some value' where id=a ;
COMMIT ;
END //
To return values from the SP, use output parameters or use the result set from the last select statement in the SP.