I ended up finding a solution for the problem myself. Because I set the ManipulationMode to control, I guess I disable some propertys which end up giving an lag when you move your object. So I ended up trying to make a storyboard when ever I move my object, which fixed the problem and gives me a nice smooth motion when I select the object I wish to move. What I did was I went into my ManDelta:
public void ManDelta(ManipulationDeltaEventArgs e)
{
Point fingerPosition = e.DeltaManipulation.Translation;
Unit.x = fingerPosition.X + ChampModelSel.x;
Unit.y = fingerPosition.Y + ChampModelSel.y;
}
and added some storyboard from the code behind, using this http://www.buzzfrog.se/index.php/2013/06/create-storyboards-in-code-for-windows-phone/ as a guide line. The ManDelta occurs every time I select the object and activate the ManipulationDelta