OpenGL 3D camera gluLookAt

Question

I'm trying to set up a 3D camera with the gluLookAt method. So I have got a 10x10x10 cube and now i want to move the camera inside that cube. I have something like that:

gluLookAt( camera->x,camera->y,camera->z, camera->eyeX, camera->eyeY, camera->eyeZ, 0, 1, 0 );

now I'm moving forward/backward:

if(GetKeyState(VK_UP) <0)
               {    
                    camera->x += sin(camera->angleX)*0.1f;
                    camera->eyeX += sin(camera->angleX)*0.1f;               
                    camera->z -= cos(camera->angleX)*0.1f;
                    camera->eyeZ -= cos(camera->angleX)*0.1f;
            }

now I'm rotating left/right:

if(GetKeyState(VK_LEFT) <0)
            {   
                camera->angleX -=0.1f;
                camera->eyeX = sin(camera->angleX) +camera->x;
                camera->eyeZ = -cos(camera->angleX) + camera->z;
            }

So that all works perfectly but now i want to rotate up/down while the SHIFT button is pressed. So i have something like that:

if(GetKeyState(VK_SHIFT) <0)
            {   
                if(GetKeyState(VK_UP)<0)
                {
                    camera->angleY +=0.1f;                  
                    camera->eyeY = sin(camera->angleY) +camera->y;                  
                }

And actually something strange happens. The camera keeps bouncing up and down all the time and slowly moving forward. Additionaly I want to add that when I look up and move forward the camera actually goes there where it looks. So basicaly the situation looks like that: I'm a ghost trapped in a 10x10x10 cube and can walk wherever I want. I want to move to the upper right corner? I JUST GO THERE. So... any ideas what should I change/add?

Answer 1

EDIT: I am starting my reply from scratch as I may have assumed too much familiarity with the subject.

The problem you are facing is that your formulas are basically not correct : your formula for rotating left/right are correct under the assumption that the "up" vector (the vector pointing upward from the camera) is always [0 1 0]... which is not the case if you also want to rotate up/down. And your formula for rotating up down is not correct since it only modifies the Y component and rotations do not work that way.

The correct way to handle that is:

• to store 3 variables that represent the camera position (as you did). Let's call them Px, Py, Pz
• to store 3 variables that represent the camera view direction (instead of your eyeX/Y/Z that encode the point the camera is looking at). Let's call them Vx, Vy, Vz
• to store 3 variables that represent the camera right vector (or up vector, as you wish). Let's take the right vector, and call it Rx, Ry, Rz.

Alternatively, you can have a nice "Vector" class that represent vectors instead of storing 3 variables each time. This is a detail at this point.

Now, your method to move your camera forward just becomes:

Px += Vx;
Py += Vy;
Pz += Vz;

You can use, for example, Rodrigues formula to rotate (hoping nobody will launch at you the "quaternion" magic word to express their cleverness ;) ). The general self-contained code to rotate around an arbitrary axis would then be:

// rotate the vector (vx, vy, vz) around (ax, ay, az) by an angle "angle"

void rotate(double &vx, double &vy, double &vz, double ax, double ay, double az, double angle) {
  double ca = cos(angle);
  double sa = sin(angle);
  double crossx = -vy*az + vz*ay;
  double crossy = -vz*ax + vx*az;
  double crossz = -vx*ay + vy*ax;
  double dot = ax*vx + ay*vy + az*vz;
  double rx = vx*ca + crossx*sa + dot*ax*(1-ca);
  double ry = vy*ca + crossy*sa + dot*ay*(1-ca);
  double rz = vz*ca + crossz*sa + dot*az*(1-ca);
  vx = rx; 
  vy = ry; 
  vz = rz;
}

And make sure to keep normalized coordinates for your camera vectors.

Now, to specifically rotate your camera up/down when you press a button:

// rotate up:
rotate(Vx, Vy, Vz, Rx, Ry, Rz, some_CONSTANT_angle);
// rotate down:
rotate(Vx, Vy, Vz, Rx, Ry, Rz, - some_CONSTANT_angle);

To rotate left/right, you first need to compute the "Up" vector that doesn't need to be stored (unless you want to, but it is redundant), and rotate both your view direction and right vectors:

// find up vector using a cross product:
  double Ux = Ry*Vz - Rz*Vy;
  double Uy = Rz*Vx - Rx*Vz;
  double Uz = Rx*Vy - Ry*Vx;

//rotate left
    rotate(Rx, Ry, Rz, Ux, Uy, Uz, some_CONSTANT_angle);
    rotate(Vx, Vy, Vz, Ux, Uy, Uz, some_CONSTANT_angle);
// rotate right
    rotate(Rx, Ry, Rz, Ux, Uy, Uz, - some_CONSTANT_angle);
    rotate(Vx, Vy, Vz, Ux, Uy, Uz, - some_CONSTANT_angle);

Setting up your camera matrix now becomes:

  gluLookAt( Px, Py, Pz, Px+Vx, Py+Vy, Pz+Vz, Rx, Ry, Rz); // or is it Ux, Uy, Uz at the end? don't remember.

Of course, I didn't test any of this code, and wrote it now. Hope it works!