Thursday, July 2, 2015

Lighting theory for 3D games, part 4: how to light a game world in a game engine

This is part of a series on how I approach game lighting, from a more general and conceptual perspective. I build most of my examples in Unity, but this is meant to be generally applicable to any 3D game engine, most of which have similar lighting tools.

We started by thinking about light from a cultural and conceptual lens in part one. In part two, we treated light more instrumentally in terms of level design and readability. Then in part three, we surveyed the three-point lighting method for use in games. But none of this theory matters if we can't actually achieve it within the semi-hard constraints of computer graphics.

Lighting is traditionally one of the slower or "expensive" things to calculate and render in a game engine. Consider the science of visible light: countless photons at different wavelengths bouncing around at unimaginable speeds that somehow enter your eye. To do any of this at a reasonable framerate, game engines must strategically simplify light calculations in specific ways, and then hope players don't notice the inconsistencies. It is "fridge logic" -- we want the player to nod along, as long as it "looks right."

Okay, so how do 3D game engines generally do lighting?

The simplest form of lighting is "ambient light", a constant default light level to apply to every model in the world, even the parts in shadow. This isn't realistic at all, so the ambient light usually serves more as a "fill light" (see part 3), especially for outdoor spaces, where it is often a dark navy color to make sure the shadows aren't terminating into pitch-black.

But ambient light applies a flat effect on every object equally, so some might say it doesn't even qualify as lighting because it does not really help us read into the depth or topology of a surface, which is often crucial for basic gameplay or wayfinding. (see part 2) ("Can I walk up this hill, or is it too steep?") ... For light to seem like light, it has to change based on the direction of the surface, and standard ambient lighting doesn't do that.

Ok, this is better. Now we're getting some shape differentiation.

"directional light" is for sunlight or any other global light source, casting a constant light from a given rotation. It affects the entire world all at once, so it can be placed anywhere, even inside a wall; only its direction matters. Again, a directional light will shine on every surface facing toward it, and distance from this sun / moon does not matter. Because most scenes and game worlds will only have a single directional light, they usually function more like "key lights." (See part 2)

A "spotlight" casts a X degrees-wide cone of light at Y intensity. If you were making a game with lots of stage lighting, recessed ceiling lighting, or street lamps, then you would probably rely heavily on spotlights. The most common use of a spotlight is an elevated spotlight pointing down, de-emphasizing the ceiling and highlighting the floor instead. In this way, spotlights are great for implying specific directions or emphasizing specific places, whatever they're shining on.

Depending on how you angle a spotlight, it's not always clear where the light is actually coming from. Use that to your advantage.

Lastly, a "point light" is an invisible omnidirectional ball of light that casts light in all directions at X intensity. You would commonly use this type for things like table lamps, cage lights, or chandeliers. Point lights are really good at drawing attention to a given light source, because they generally have to be near whatever they're illuminating and all nearby shadow-casting objects will practically be pointing to it. (Versus a spotlight, where it's not really clear where the light is coming from, and we don't really care either.)

Together, these 4 light types are generally found in every 3D game engine or rendering package. They sort of form a complete domain:

3D LIGHT TYPES: Global, affects everything Local, affects nearby things
Shines in one direction Directional light Spotlight
Shines in all directions Ambient light Point light

But really, this is just the beginning.

Direct lighting only accounts for the first ray of light, which is why it's called "direct lighting", as opposed to "indirect lighting" which attempts to model how light bounces off or interacts with surfaces. In the right-most image above, notice how the red from the triangle bleeds onto the floor, and how the blue from the cube bleeds into the shadow on the wall, and how the background is just generally brighter. Those effects come from light splashing around this space, and traditionally it has been very expensive to try to calculate these effects in real-time while also rendering your favorite face-shooting game at 60 frames per second. (I'll talk a lot more about this in a future installment.)

Try holding your hand up in front of a bright light source -- the red fringe around the silhouette of your hand is called "subsurface scattering", because the light is actually passing below the surface of your skin and bouncing off the blood. We could say that the standard direct lighting model does not account for different material or shading types, so that means we have to engineer a system to account for that... and what about glare, and/or the ways your eyes eventually adjust to darker or brighter spaces? We'll have to engineer those systems too!

So, this is basically the technical history of game lighting: trying to compensate for all the holes within this basic lighting model. Because of this, the 4 basic light source types are just the beginning of game lighting. There's so much more we have to think about and implement:

Let's say you're making a game set in a city at night.

First, start with your global settings. My ambient light is set to a dark blue-ish purple, and I have a directional light casting a surprisingly bright blue light on the whole scene. (see "Hollywood Darkness": We only want it to feel dark, not actually be dark.)

Then start adding in your local lights. In this case, there's a spotlight as well as a faint point light underneath to simulate light bouncing and splashing around.

Now let's start adding things that aren't "lights", but strongly affect how we perceive the light.

The fixture model now has a self-illuminated emissive material to appear bright. The light halo is a separate 2D sprite that fades in and out based on the player camera distance. There's a particle system spraying dust particles in a cone shape beneath the light. I've also applied a bloom, ambient occlusion, and chromatic aberration effect to the camera. And don't forget the skybox, which is the only way "Hollywood darkness" can really work. Lastly, I added some fog in the background to simulate some darkness / atmospheric scattering and push back the scene background a bit more.

Remember that the perceived brightness of a light depends on two things: (1) the actual brightness of the light, and (2) the relative light level / darkness in the areas around that light. In general, try to darken or dim unimportant things so the more important things can get more attention. Context matters!

Specularity and rim-lighting and self-illumination, projected shadows, chromatic aberration, screen-space ambient occlusion, cubemaps and spheric harmonics, high dynamic range tonemapping, glare and light halos... in game engines these technologies operate as separate features that are selectively enabled or disabled to optimize a game to reach a certain framerate or achieve a certain art style, but in real-life these are all part of the unified phenomena of light as we know it.

In the end, the player will only see one light source there. But we'll know better, we'll be able to see the subtle system of effects we've concocted...

Game lighting is not a unified system. Rather, it is a patchwork of all this random shit that will hopefully seem to go together. As designers, it's our job to make all these different effects and hacks seem like a coherent thing.

Sometimes this control is really nice when you can, for example, create invisible floating light sources without any visible light fixtures. (Paraphrasing my lighting design teacher: "I would kill for that kind of power.") But sometimes all this responsibility kind of sucks because you have to go and manually add every part and fine-tune it with all the other parts, and it can take a lot of work. (This is why many triple-A studios have now taken away time-consuming lighting duties from level designers, and given this work to environment artists and/or dedicated lighting artists. For more info on a history of level design, see my GDC 2015 talk.)

Lighting in games is about more than just lights. It's about the overall image and what kind of mood you want to build, while seeming plausible enough within your world that the player doesn't constantly double-take. But this idea of "plausibility" is dependent on notions of "what is realistic" -- which affects what types of new lighting technologies get developed, which affects what is aesthetically possible in our toolsets, which pushes for more "realism"... and so on.

Next time, I'm going to unpack some of these underlying assumptions about what "good game lighting" is, and how we can possibly transcend these assumptions, if at all.

NEXT TIME: part 5, the cult of fidelity.