Retrieving the actual 3D coordinates of a point on a triangle that has been flattened to 2 dimensions

Question

This is a bit of a complicated problem, so I'll do my best to break it down into chunks.

I'm writing a 3D Python library for the sake of learning / fun (as opposed to one that I'd intend for others to use). In the system I've developed, three-dimensional points are generally flattened to the image as follows:

• Increasing the Z index by width moves the point halfway to the vanishing point in the center.
• At Z = 0, the X and Y values correspond directly to the pixel at X, Y.

(There might be a name for this method, but if there is, I'm not familiar with it.)

In Python:

# vx and vy are the vanishing point's coordinates
def flatten_point(width, vx, vy, x, y, z):
    distance = (x - vx, y - vy)
    flat_distance = [d / (1 + float(z) / width) for d in distance]
    return (vx + flat_distance[0], vx + flat_distance[1])

At this point, I'm able to create triangles somewhat efficiently by flattening its vertices and using barycentric coordinates to find and fill in the pixels that fall between those three points. This works well enough if I don't need to know anything about the actual points on the triangle that those pixels correspond to, but if I want to shade the triangle so that deeper points are drawn darker, I need to know what unflattened point on the triangle the pixel corresponds to.

joriki on math.stackexchange recommended using the barycentric coordinates as weights to find the original point. This did appear to work for awhile -- and it probably would work if I were using a linear depth system -- but it falls apart when the depths of the triangle's points differ by enough. The triangle appears to approach the greatest depth more quickly than it actually does, as if it were curved backwards.

So, in short: how can I reverse the point flattening function to get the actual 3D point of an arbitrary 2D pixel on a flattened triangle? Alternatively, if there is a better / more efficient way to flatten triangles without losing the depth of each pixel, that would work too.

Answer 1

You are right that the problem lies in your depth values not being linear. Fortunately, the solution is simple, but a little expensive if calculated per pixels.

Using your barycentric coordinates, rather than interpolating the three Z components directly, you need to interpolate their inverse and reinverse the result. This is called perspective correction.

Example for Z only :

def GetInterpolatedZ(triangle, u, v):
    z0 = 1.0 / triangle[0].z
    z1 = 1.0 / triangle[1].z
    z2 = 1.0 / triangle[2].z
    z = z0 + u * (z1-z0) + v * (z2-z0)
    return 1.0/z

With triangle a list of three vectors and u and v the barycentric coordinates for triangle[1] and triangle[2] respectively. You will need to remap your Zs before and after the divisions if they are offset.

If you want to interpolate the actual X and Y coordinates, you do something similar. You will need to interpolate x/z and y/z and relinearize the result by multiplying by z.

def GetInterpolatedZ(tri, u, v):
    t0 = Vec3(tri[0].x/tri[0].z, tri[0].y/tri[0].z, 1.0/tri[0].z)
    t1 = Vec3(tri[1].x/tri[1].z, tri[1].y/tri[1].z, 1.0/tri[1].z)
    t2 = Vec3(tri[2].x/tri[2].z, tri[2].y/tri[2].z, 1.0/tri[2].z)

    inter = t0 + u * (t1-t0) + v * (t2-t0)
    inter.z = 1.0 / inter.z
    inter.x *= inter.z
    inter.y *= inter.z
    return inter

Again, tri is a list of the three vectors and u, v are the barycentric coordinates for tri[1], tri[2]. Vec3 is a regular 3 components Euclidean vector type.