Lost with skinned meshes

Question

I'm kind of lost,

I finally got time to work on skinned mesh with MD5 on GPU - through matrices, and I desparately can't get correct result (it is so damn wrong)!

So here is what I do:

• Load the MD5MESH file

• Build bind-pose mesh (this one step is done correctly - the geometry is correct)

• Create bindpose and inverse bindpose like this (note the joints are used as loaded, not transformed by parent, do they need to be?):

      for(unsigned int i = 0; i < this->mMatricesCount; i++)
  {
      mat4 mBoneTranslation = mat4(
              1, 0, 0, mModel->mJoints[i].mPosition.x, 
              0, 1, 0, mModel->mJoints[i].mPosition.y, 
              0, 0, 1, mModel->mJoints[i].mPosition.z, 
              0, 0, 0, 1);
      mat4 mBoneRotation = mat4(mModel->mJoints[i].mOrientation);
  
      mat4 mBoneMatrix = mBoneTranslation * mBoneRotation;
  
      this->mMatrices[i] = mBoneMatrix;
      this->mInverseMatrices[i] = inverse(mBoneMatrix);
  }
  

• Load MD5ANIM file (where each frame joints are computed as):

                  if(mAnimation->mJoints[i].mParent < 0)
                  {
                      mAnimation->mFrames[mFrameID][i].mPosition = _position;
                      mAnimation->mFrames[mFrameID][i].mOrientation = _orientation;
                  }
                  else
                  {
                      MD5FrameJoint *mParent = &mAnimation->mFrames[mFrameID][mAnimation->mJoints[i].mParent];
  
                      float4 rpos = rotate(mParent->mOrientation, _position);
  
                      mAnimation->mFrames[mFrameID][i].mPosition = rpos + mParent->mPosition;
                      mAnimation->mFrames[mFrameID][i].mOrientation = mParent->mOrientation * _orientation;                           
                  }
  

• Each frame, build Bone matrices like (that maximum of 4 weights right now is NOT causing the trouble, as currently I have just 3 bones in scene on simple tessellated quad - rotates wrongly):

  for(unsigned int i = 0; i < this->mJoints; i++)
  {
      const mat4 mTranslate = mat4(
          1, 0, 0, this->mFramePositions[mFrame][i].x, 
          0, 1, 0, this->mFramePositions[mFrame][i].y, 
          0, 0, 1, this->mFramePositions[mFrame][i].z, 
          0, 0, 0, 1);
      const mat4 mRotate = mat4(this->mFrameOrientations[mFrame][i]);
  
      this->mOutput[i] = mTranslate * mRotate;
  }
  

• And calculate vertices like (doing on CPU now, would like to move it to GPU):

          for(unsigned int j = 0; j < mSkinnedModel->mVertexCount[i]; j++)
          {
              float4 mResult = float4(0, 0, 0, 0);
  
              float4 mPosition = float4(mSkinnedModel->mVertices[i][j].mPosition[0],
                                        mSkinnedModel->mVertices[i][j].mPosition[1],
                                        mSkinnedModel->mVertices[i][j].mPosition[2],
                                        1.0f);
  
              for(unsigned int k = 0; k < 4; k++)
              {
                  mResult += (mAnimatedBones[mSkinnedModel->mVertices[i][j].mBoneIndices[k]] * mPosition) * mSkinnedModel->mVertices[i][j].mBoneWeights[k];
              }
  
              mBuffer[j].mPosition[0] = mResult.x;
              mBuffer[j].mPosition[1] = mResult.y;
              mBuffer[j].mPosition[2] = mResult.z;
              mBuffer[j].mPosition[3] = 1.0f;
          }
  

Mesh + anim files are correct (exported them and imported to 3d modelling software, works!)

Right now I'm testing whether my math library is okay and so far it seems good (matrix inversion is good, quaternion multiplication too, matrix from quaternion tested, ...)

And please don't point me to any articles, I've got some of them (incl. those with MD5 skinning on GPU with matrices + source code) opened and figuring out whats wrong, so far the code seems almost exactly like mine. It will be some tiny little crappy detail.

Anyone see where the guru is?

Answer 1

(it is so damn wrong)!

One picture is worth thousands of words. You didn't post the picture. So, what did you expect?

 mat4 mBoneTranslation = mat4(
        1, 0, 0, mModel->mJoints[i].mPosition.x, 
        0, 1, 0, mModel->mJoints[i].mPosition.y, 
        0, 0, 1, mModel->mJoints[i].mPosition.z, 
        0, 0, 0, 1);

Is this supposed to be OpenGL matrix? OpenGL matrices have different orientation, and their layout is completely identical to D3DXMATRIX/D3DMATRIX used in DirectX - unless you want to store all matrices transposed.

Non-transposed translation matrix:

 mat4 translation = mat4(
        1, 0, 0, 0, 
        0, 1, 0, 0, 
        0, 0, 1, 0, 
        x, y, z, 1);

And of course, non-transposed matrices use reverse multiplication order:

mat4 combined = rotation * translation;

Also, instead of multiplying, you can simply copy last row (non-transposed):

mat4 combined = rotation;
combined.rows[3] = translation.rows[3]; 

Or column (transposed).

Also, in this part:

 this->mOutput[i] = mTranslate * mRotate;

You forgot to include inverse bone transform into calculation. Withotu inverse bone transform skinned mesh will "blow up".

It should be (transposed matrices)

 this->mOutput[i] = mTranslate * mRotate * mInverseMatrices[i];

or (for non-transposed).

 this->mOutput[i] = mInverseMatrices[i] * mRotate * mTranslate;

---

--EDIT--

It is possible that you're incorrectly calculating joint hierarchy transforms.

{
    MD5FrameJoint *mParent = &mAnimation->mFrames[mFrameID][mAnimation->mJoints[i].mParent];

    float4 rpos = rotate(mParent->mOrientation, _position);

    mAnimation->mFrames[mFrameID][i].mPosition = rpos + mParent->mPosition;
    mAnimation->mFrames[mFrameID][i].mOrientation = mParent->mOrientation * _orientation;                           
}

That's not how it is normally done.

Calculate local transform for every joint.

this->localTransform = positionMatrix * rotationMatrix

Calculate world transform for every joint, by multiplying parent transform and child transform

this->worldTransform = parent->worldTransform * this->localTransform;

For root node world transform is multiplication of object matrix (that translates mesh around) and local transform.

root->worldTransform = worldMatrix * root->localTransform;

Your scheme will not work properly if matrix (ANY matrix) in hierarchy has scaling component.

If this doesn't help, then you'll need to get somebody to debug your code. I don't do debugging for free, but it is not allowed (AFAIK) on SO to offer "freelancing" services. So you'll have to find somebody else.

Another thing is that instead of using "white dots to indicate \"where it should be\"", you must use application that can display animated md5 format AND is guaranteed to do it correctly. There's no warranty that your white dots are at correct positions otherwise.