I think you want to move the camera on a sphere, rather than a plane as you are doing now.
Rather than storing the position of the camera in Euclidean coordinates, store it in spherical coordinates.
double distance;
double latitude;
double longitude;
In the mouse wheel handler, simply increase/decrease distance
.
In the mouse move handler, adjust latitude
proportional to the Y delta (you might want to limit it to +/-pi/2, so that you don't overshoot the poles), and longitude
proportional to the X delta.
To calculate your position, use something like this:
Point3D position = new Point3D
(
Math.Cos(longitude) * Math.Cos(latitude) * distance,
Math.Sin(latitude) * distance,
Math.Sin(longitude) * Math.Cos(latitude) * distance
);
// This line is just a simplified version of your LookDirection calculation
Vector3D lookDirection = objectPosition - position;