You can fix this by checking out your feature branch and doing:
git rebase --onto dev HEAD~<number of commits you care about> --force-rebase
This instructs git to replay the commit range that you specify and to ignore whether or not the commit occurred at a point in time prior to the commits on dev.
Note that this will leave you with a history that looks like:
<did some work> -- <reverted all of that work> -- <re-did that work>
which may not be all that useful to you. Some of the other answers here detail how you can get a cleaner history by rewriting it, depending on your team's tolerance for history rewriting.