Have a look at ? ave
, it does exactly what you are looking for:
remove_outliers<-function(x){
qnt<- quantile( x,probs=0.99 )
x[ x>qnt ]<- NA
return(x)
}
# assuming your data.frame is called mdf
mdf$fixed <- ave( mdf$share, mdf$group, FUN = remove_outliers )
mdf
group share fixed
1 A 100 NA
2 A 50 50
3 A 30 30
4 A 10 10
5 B 100 NA
6 B 90 90
7 B 80 80
8 B 60 60