Open topic with navigation

# Introduction to Stereoscopic 3D

This primer introduces some key concepts for adding stereoscopic 3D to your application. We define basic terms that are used throughout the SDK documentation. Even if you are familiar with 3D graphics, you should read this primer.

## Traditional Rendering: The Monoscopic 3D Viewing Frustum

Traditional 3D images are rendered from only one perspective, so we call them monoscopic or one-eyed. If you have worked with 3D graphics before, for example in games, you may be familiar with monoscopic 3D. The rendered scene may have depth, but the image looks the same no matter where you move your eyes. Everything rendered in a monoscopic image comes from a pyramid-shaped volume called a frustum. The frustum is formed by tracing planes from the viewpoint through the four edges of the screen.

- The tip of the pyramid is the viewpoint (the eye or camera). This is typically placed on a line perpendicular to the center of the display, resulting in a symmetric frustum.
- The base of the pyramid is called the far clipping plane.
- Somewhere between the viewpoint and the far clipping plane, there is a near clipping plane, which effectively chops off the top of the pyramid.

Figure 1 shows a monoscopic 3D frustum, with the viewpoint and clipping planes. The angle between the top and bottom walls of the frustum is called the vertical field of view or FOVY. Similarly, the horizontal field of view or FOVX is the angle between the left and right walls. The aspect ratio of an image is the relationship between its horizontal and vertical sizes. The aspect ratio determines the shape of the rendered image. For a symmetric frustum, the aspect ratio is equal to the fraction FOVX/FOVY.

Figure 1: The Monoscopic Viewing Frustum

During rendering, the contents of the frustum are effectively projected onto the near clipping plane, resulting in a flat image. Each point in the projected image corresponds to what you would see if you put your eye at the viewpoint and looked along a line through that point on the near clipping plane.

## Stereo Principles

Stereoscopic rendering creates the illusion of depth by presenting two separate images to the viewer's left and right eyes. These two images are slightly offset, as though you were looking through binoculars and closing one eye at a time. The brain fuses the two images into a single 3D image. Note that the two separate images must closely correspond to the images the eyes would see when looking at the real object.

Note:
  Refer to these links for more information on 3D imaging:
  - Stereoscopy
  - Stereopsis

## Depth Perception in zSpace

Your brain uses parallax information to locate an image in space. Parallax refers to an object appearing slightly different in perspective when viewed from different angles or positions. Binocular parallax refers to viewing the world around us with two eyes that are about 60mm apart. Another important form of parallax, motion parallax, results when you move your head around.

zSpace uses both forms of parallax to create a strong sense of depth. It presents a different high-definition image to each of your eyes in real time. Each image looks correct from your eye's current position. Figure 2 illustrates depth perception in zSpace. Notice the differences between the left and right views of the object.

Figure 2: Depth Perception in zSpace

### Positive and Negative Parallax

Whether an image appears to be on the screen, behind the screen, or in front of the screen depends on the relationship of the left-eye and right-eye images to each other.

- An object appears to be directly on the screen when its images for the left and right eyes overlap exactly. This is known as zero parallax because the object will look the same no matter where you move your eyes. A zero-parallax image will even look correct if you take off your 3D glasses.
- When an object is moved behind or "inside" the screen, the left eye's image appears to the left of the right eye's image. This is positive parallax.
- When an object is moved in front of the screen, close to the user, the left eye's image appears to the right of the right eye's image. This is negative parallax.

Figure 3 illustrates parallax in zSpace.

Figure 3: Parallax Regions

## The Stereo 3D Viewing Frustum

Stereoscopic rendering is similar to monoscopic rendering, but the same object is rendered from two separate viewpoints, one for each eye's view. There are some other important differences too.

1. Each viewpoint is not on a line perpendicular to the center of the display. This is known as an "off-axis stereo projection" and it results in two asymmetric frusta.
2. As a result, the near and far clipping planes are different for the two frusta and their aspect ratio is not simply the ratio between the horizontal and vertical fields of view.
3. The vertical and horizontal fields of view are determined by the two viewpoints. The closer the viewpoints are to the display, the wider the fields of view become.

Figure 4 shows the stereo frustum, with one off-axis frustum for each eye's viewpoint. Note the object disappears as it crosses the near and far clipping planes.

Figure 4: zSpace Stereo Frustum

The frusta overlap exactly at the screen. As previously discussed, if we render anything at exactly the same location as the screen, it will appear the same in both eyes. For this reason, we say the screen is on the zero parallax plane. The screen lies somewhere between the parallel near and far clipping planes.

The viewer's distance from the screen directly affects the stereoscopic 3D frustum. The stereo frustum also depends on the angle of the display with respect to the viewer, the size of the display, and the distance between the eyes of the viewer. The Stereo Frustum API correctly calculates the stereo frustum and associated rendering matrices based on these factors.

zSpace uses many perceptual signals to convey a strong sense of realism. To create a correct stereo 3D image, the key is to reproduce as closely as possible the actual views seen in each eye. Using incorrect image offsets will result in improper depth alignment in the stereoscopic 3D image. This can lead to eye strain and other undesirable effects.

## Key Parameters

Here are the main factors the Stereo Frustum API takes into account when computing a stereo viewing frustum.

### Inter-Pupillary Distance

Inter-Pupillary Distance (IPD) refers to the distance between the viewer's two pupils. IPD is one of the key parameters that control comfortable viewing of stereoscopic images. Using an incorrect IPD will present images to the viewer's eyes that do not match the viewer's real world experience. The average adult IPD is approximately 63 mm. The majority of adults have IPDs in the range 50-75 mm while the wider range of 45-80 mm is likely to include almost all adults. The minimum IPD for children is around 40 mm.

### Head Pose

There are two ways to calculate viewpoints for the viewer's eyes: static and dynamic.

- In the static case, the viewer's location is fixed and he or she must remain at that location for proper stereoscopic viewing. This location is known as the default head pose.
- In the dynamic case, the viewer's location is continually monitored and the offset from the standard viewing location is relayed back to the system in real-time. This enables the zSpace system to convey motion parallax.

zSpace supports both static and dynamic viewer locations.

## Head Tracking

The zSpace System tracks your head to enable motion parallax, an important cue for depth perception. The polarized glasses have two purposes:

- They filter light to separate the images seen by your left and right eyes.
- They include markers for tracking the viewer's head position. This allows the application to redraw the image based on the viewer's current perspective. zSpace head tracking uses a number of techniques to provide smooth and responsive tracking.

zSpace uses the current eye positions to recompute the off-axis frusta every frame. Use the zSpace SDK's APIs to control this by one of the following two ways:

      // Automatically update all created frustums with the default head pose.
      ZSContext zSpaceContext = ...;
      zsUpdate(zSpaceContext);

OR

      // ZSContext set during initialization.
      ZSContext zSpaceContext = ...;

      // Grab a handle to the stereo frustum.
      // Implementation omitted for brevity.
      ZSHandle frustumHandle = ...;

      // Grab a handle to the default head target.
      ZSHandle headTargetHandle = NULL;
      zsFindTargetByType(zSpaceContext, ZS_TARGET_TYPE_HEAD, 0, &headTargetHandle);

      // Grab the head target's pose.
      ZSTrackerPose headPose;
      zsGetTargetPose(headTargetHandle, &headPose);

      // Manually update the frustum's head pose.
      zsSetFrustumHeadPose(frustumHandle, &headPose);

Note:
  The viewer can freely look around objects in zSpace and see behind them. If the viewer changes perspective, for example by tilting the head, the object is rendered just as one would see it in the real world.
  When performing head tracking, we take rotation of the viewer's head, thus the orientation of his or her eyes, into account. Without this, the image would always appear as if the viewer looked directly at the zSpace with his or her eyes being parallel to the display. This would lead to an uncomfortable viewing experience.
  Head tracking and off axis frustums are changed each frame to give correct perspective. By updating the frustums each frame of the scene's duration, we create a natural and comfortable experience. Without updating the frustums for each frame, we would not be performing proper head tracking and that would lead to a suboptimal viewing experience.

Open topic with navigation