Silhouettes, Outlines, Feature Edge Detection, whatever you call it. There are a dozen ways to do it and none of 'em are pretty, so I've been delaying the decision. We want to get early builds out to playtesters in September, however, and the game isn't exactly playable without silhouettes, because in some places you can't see the door you need to open if it's on a south/east wall. You can move your mouse around until you see "Door" but that's lame.

So, I did some more research and knocked out a prototype this afternoon:

The 'obscured' version we'll use for hidden objects:

Technical rundown

This is what I'll call the 'backface method'. It's an old one, often used for cel shading in 3D games.

>In a nutshell, simply redraw the object's BACK faces, scaled out ~1% along the vertex normals, in the desired color.

• You can flip the normals if you're using backface culling.
• Or use front-face culling: glEnable(GL_CULL_FACE); glCullFace(GL_FRONT);
• Use a bare-bones shader. All you need is a solid color fill. Very efficient.
• Vertex scaling: vertex = vertex + .01*normal;
• Raycast from camera to determine if an object is obscured. If so, render outline with depth test off. This draws a solid fill, so just render it again in black, unscaled.

Other methods:

• Triangle edge detection - http://ogldev.atspace.co.uk/www/tutorial39/tutorial39.html
  • Draw edges between each pair of adjacent triangles where one is facing back and the other facing front
  • Downside: requires vertices in GL_TRIANGLES_ADJACENCY format; no big deal but I'd rather not
  • Downside: wide lines semi-required, and that's more work in GL3+ and D3D
  • Probably requires too much fussing with thresholds, depth offsets, etc
• Stencil method - http://www.flipcode.com/archives/Object_Outlining.shtml
  • Render to color+depth+stencil buffer, then render wireframe where stencil is unset (outline will be half width of wireframe lines)
  • Downside: requires wide lines; suggests http://flipcode.com/archives/Textured_Lines_In_D3D.shtml
  • Stencil buffer hacks are cool but why use it if you don't have to?
• Fragment/pixel shader methods
  • Draw pixels in the outline color when almost-perpendicular to camera [dot(normal,camera) < .01]
  • Downside: doesn't work with sharp-edged models, of which we have many
• Fake it with a wireframe
  • Good enough for at least one commercially successful game I've played this year!
  • Downside: hardly any easier than backface method