Receiving GLU_TESS_ERROR_5 from GLU Tesselator
https://stackoverflow.com/questions/3411570
italiano
english
français
española
中国
日本の
العربية
Deutsch
한국어
Português
RussianQuestion
Here is my issue. I'm tesselating complex, self intersection, multicontour polygons with hundreds of verticies. The GLU Tesselator crashes with null pointer 0x0000000 issue. It never ever crashes when I do not make self intersecting polygons. If it does not intersect, it will never crash no matter what the circumstances. I check for NULL EVERYWHERE in my application, I'm sure it's not on my side of things. I found an old version of GLU 1.2 from SGI and it never crashes, however the mesa and Windows versions based on GLU 1.3 both crash. Nothing crashes in debug mode strangly enough. To get more information I compiled Mesa's GLU and saw that first an assert fails, then if I comment that out, there is a pointer which is set to NULL from a function which fails to malloc. I'm very unsure at this point what to do. What could I do to try to solve this issue? Should I just try to make a version of Mesa's GLU which works for me? I'm just unsure how to proceed from here.
After more debugging I see I'm getting GLU_TESS_ERROR_5 which I think is a number too large error but I did a for loop to test for numbers greater than that but no luck :(
Solution
At least on Windows, GLU_TESS_ERROR_5 means that one of the coordinates was too large. Specifically, GLU requires that the coordinates are small enough to be multiplied together without overflow. The specification says that the limit is defined in the constant GLU_TESS_COORD_TOO_LARGE. If this constant exists, check that the absolute value of every coordinate is less than it. If not, I think it would be safe to check that the coordinates are between - 10^150 and 10^150. If that doesn't work, try progressively smaller ranges.
It may also be that there's a problem with geometry, which triggers another problem in GLU. Try to find the simplest polygon that will trigger this error.
If that doesn't work, see if there is a newer version of GLU available. I don't know about Mesa, but the version of OpenGL shipped with VC++ is notoriously out of date.
If all else fails, you could try using another library to perform the tesselation. After a quick search, Triangle1 appears to be a good candidate.
