Projection Geometric Distortion Compensation

Kevin Singleton

TAMU Visualization Laboratory

Introduction

For virtual environment systems, each face, or facet, of display geometry is rendered on a computer then projected onto a physical model of the facet. However, distortions in the lens can change the shape of the rendered facet that is being projected. The projected image would not correctly align with the physical model even though they are based on the same measurements. Therefore, a means of changing, or warping, the rendered facet is needed to adjust for lens distortions so that both projected and physical facets align.

Design Approach

Code for the facet warping ability was integrated into an existing cave application called the 3Dengine. The facet warping code was added as a separate object on top of the existing code, thereby not modifying the existing code substantially.

Facet warping needs to be reusable. For a given display model, the warp is remembered from previous uses of the 3Dengine software. Later executions of the application require less modification to the facet shape to account for projector lens distortion.

Implementation

A projected facet of the display geometry is broken into a grid of smaller polygons. Each corner of the smaller polygons is a vertex, which has a defined position and texture coordinate. For 3Dengine applications, control vertices (CVs) are those vertices on the outer edges (sides) of the facet that help the user manipulate the overall shape of the facet. The positions of the vertices that are not CVs are automatically changed when a control vertex is modified. Texture coordinates are based on the unwarped rendering of the display facet. Each frame of the rendered facet is stored into a texture that is the same size as the screen resolution. As the position of the control vertices changes, the texture coordinates remain static and the texture is stretched or shrunk, creating the look of a warped image.

The initial warp vertices are obtained from reading in data from a user-supplied file. This file must have a .warp extension. Vertex data in the warp file can have two different structures: a weighted distribution or per-vertex.

Weighted distribution [Appendix A] is the most general file structure and is used when utilizing the warp functionality on a display facet for the first time. For each side of a display facet, the control vertex position is determined by linearly interpolating between the current two corners with the total weight summed thus far. For example, if a side has a distribution of [ .2 .3 .3 .2 ], the side will have 5 CVs (including the 2 corners), with the first CV at one corner, the second CV at 20% of the distance from the first corner to the second corner, the third CV at 50% of the distance from the first corner to the second corner, etc. Notice, the sum of the weights for each side must total 1. Texture coordinates can be calculated by using screen coordinates.

The per-vertex file structure [Appendix B] looks similar to .obj file format. Each vertex (control vertices along the side of a facet and the vertices in the middle of the facet that are not user-controlled) is represented by a ‘v’ followed by three floats representing the vertex position in cave object space (although the z-coordinate is the same for all vertices). The texture coordinates for each vertex are represented by a ‘vt’ followed by two floats. The per-vertex file structure is automatically written when the 3Dengine application is terminated. This saved file structure makes less modification to the facet shape necessary when the application is run again using the same display geometry.

Warping the display facet requires selecting the CV that the user wants to modify and repositioning it with the arrow keys on the keyboard. Traversing the warp control vertices follows a specific hierarchy from higher order to lower order vertices. Corners represent the highest order in the hierarchy since they define the general shape (roughest detail) of the display facet. The CVs closest to the midpoints of the facet sides are the next highest order. Further orders of control vertices radiate from the CVs closest to the midpoints towards the corners. Lower orders of CVs control the finer detail of the facet shape. When a CV is moved, the moving CV also influences other surrounding CVs. CVs with a lower order than the current CV are repositioned so that they are collinear with the current CV and the nearest corner vertex. Vertices moving towards the center of the facet move some fraction of the movement of the current CV. To simulate the curvature of the projector lens, vertices towards the center of the facet move less than vertices near the edges of the facet.

For each frame, the rendered scene is saved into an OpenGL texture buffer that is large enough to hold each pixel of the display screen. OpenGL requires textures to have dimensions that are a power of 2, and both the width and height of the texture are must be equivalent. For example, if the screen resolution is 1280 x 1024, a 2048 x 2048 texture must be used to store the rendered scene (even though the rendered scene only takes up a portion of the entire texture). After the scene is rendered to a buffer, the warping calculations take place. Each of the smaller polygons that create the facet is drawn as a polygon into the render buffer. The rendered scene texture is then mapped onto each polygon using texture coordinates that were either specified in the warp file or calculated from the original shape of the facet and warp grid. Images of the facet before and after the warping process are shown in Appendix C.

When the 3Dengine application terminates, the current positions (in display object space) and texture coordinates of all vertices are written to the warp file specified at the initialization of the application.

Performance

Using the facet warping functionality decreases overall frame rate because each frame must be rendered twice. Performance tests were run on 2.4 MHz dual-processor machines with 2048 MB RAM and nVidia Quadro4 900XGL 128 MB video cards. The performance, measured in frames per second (FPS), of three scenes with varying number of polygons was tested on a machine with a 1280 x 1024 screen resolution (2048 x 2048 texture). The same three scenes were then tested on a computer with a 1024 x 768 (1024 x 1024 texture) screen resolution. The results of these tests are shown in Table 1.

Screen Resolution: 1280 x 1024,
7 x 7 Warping Grid
Polygons in Scene	No Warp FPS	Warp FPS	FPS diff	% Drop
25,000	57.96	39.44	18.52	31.95%
50,000	22.8	19.24	3.56	15.61%
100,000	7.15	6.76	0.39	5.45%


Screen Resolution: 1024 x 768,
7 x 7 Warping Grid
Polygons in Scene	No Warp FPS	Warp FPS	FPS diff	% Drop
25,000	142.96	110.67	32.29	22.59%
50,000	52.22	46.11	6.11	11.70%
100,000	14.42	13.91	0.51	3.54%

Table 1: Frame Per Second (FPS) comparison of 3 scenes of varying complexity

For low polygon count scenes, the FPS drop is a considerably higher percentage than for higher polygon count scenes. The warping algorithm takes a fairly constant amount of time for a given grid size and screen resolution, but the initial rendering of the unwarped scene takes a longer amount of time for larger polygon count scenes (no culling occurs). The warping process takes less computation time in proportion to the initial render time as the number of polygons increases in a scene.

Frame rate decrease is less for lower screen resolutions. This is because fewer pixels must be transferred from the render buffer to the texture buffer per frame.

For most scenes, the loss in frame rate is not noticeable. Even though low polygon scenes can lose around 35% of the unwarped frame rate, the low compute times of the initial rendering keep the frame rates above 30 FPS (video frame rate). However, as more functionality is added to the 3Dengine software, the number of frames lost to warping might mean sacrificing in other areas such as rendering shadows or creating scenes with high-polygon counts.

Conclusion and Future Work

The facet warp provides a way to modify the shape of a projected display facet to adjust for projector lens distortion. The user can set the number of control points he/she wants to utilize for manipulating the shape of the facet through the warp file. Particular projector and display facet combinations can be saved so that less modification is needed to account for lens distortion on successive executions of 3Dengine applications.

Currently, the facet warp works only for quadrilateral faces. The 3Dengine has no limit on the number of sides to a facet. The warping utility should be extended to include any number of sides on a face. Also, the warp file assumes all faces of the display geometry have the same number of sides. The warp file specification will be expanded to fit the need for facets with varying number of sides.

When positioning warp control vertices, vertex movement is along xy screen coordinate axes. Future plan is for the control vertices to move along the line created by the current CV and the neighboring CV in the direction of the intended movement.

When warping is enabled, the screen is rendered to a buffer, the buffer is read into a texture, then the texture is applied to the smaller polygons that make up the facet and rendered to the same buffer. Newer versions of OpenGL are able to render directly to a texture, which would eliminate the first step and speed up rendering.

Appendix A

Weighted Distribution Warp File Format (*.warp)

# Quadrilateral with 7 x 7 warp grid

Grid[4 7]

[[ 0.05 0.1 0.15 0.4 0.15 0.1 0.05 ],

[ 0.05 0.1 0.15 0.4 0.15 0.1 0.05 ],

[ 0.05 0.1 0.15 0.4 0.15 0.1 0.05 ]]

Appendix B

Per-Vertex Warp File Format (*.warp)

# Quadrilateral with 7 x 7 warp grid

Grid[4 7]

[[ 0.05 0.1 0.15 0.4 0.15 0.1 0.05 ],

[ 0.05 0.1 0.15 0.4 0.15 0.1 0.05 ],

[ 0.05 0.1 0.15 0.4 0.15 0.1 0.05 ]]

# Vertex coordinate positions in cave coordinates

v -0.541195 -0.466974 -1.220263

v -0.487076 -0.466974 -1.220263

v -0.378837 -0.466974 -1.220263

v 0.298402 0.397570 -1.220263

v 0.378529 0.373314 -1.220263

v 0.418593 0.361186 -1.220263

# Vertex texture coordinates

vt 0.565225 0.000000 1.000000

vt 0.539952 0.000000 1.000000

vt 0.489407 0.000000 1.000000

vt 0.173154 0.403720 1.000000

vt 0.135737 0.392393 1.000000

vt 0.117028 0.386730 1.000000

Appendix C

Image 1: Unwarped display facet

Source: 3Dengine software, Texas A&M University Visualization Laboratory

Image 2: Warped display facet