In last two weeks, I was bothered by the visibility problem of dojox.gfx3d library. Eventually, I decided to review the Computer Graphics 101, and checked out Fundamentals of Computer Graphics, second edition from the library. This books poses two approaches: Z Buffer and Binary Space Partition(BSP).
The idea is for any polygon, for example, (a, b, c), the space is divided by the plane into two space: plus and minus. For any arbitrary vertex p, if (p - a) dotProduct n = 0, p is inside the plane, > 0, p is in the plus space, vice versa for minus space. n is the normal vector that is perpendicular the plate, defined as:
n = ( b - a ) crossProduct ( c - a )
When Viewport render 3D objects, it iterates all its children and apply the camera transform to map the 3D objects to the new coordination system, then build the BSP tree based upon the polygonized vertices. At the end, in-order-transverse the BSP tree to draw the 2D shape to the canvas.
So far, this algorithm works for triangle, and theoretically works fine for all polygons. we may need to consider how to bring the Edge to the table, and I do believe the Scene needs to be refactored unless some constraints are imposed to the Scene.
Here is the snapshot of the latest working copy.