This is a journey into the code the
visual system uses to work out how far away things are and how fast
they are moving. Both of the variables—depth and
velocity—can be calculated by comparing measurements of object
position over time. Rather than have separate neural modules to
figure out each variable, performing the same fundamental processing,
the brain combines the two pieces of work and uses some of the same
cells in calculating both measures. Because depth and motion are
jointly encoded in these cells, it's possible (under
the right circumstances) to convert changes in one into changes in
another. An example is the Pulfrich Effect, in
which a moving pendulum and some sunglasses create an illusion of the
pendulum swinging in ellipses rather than in straight lines. It works
because the sunglasses create an erroneous velocity perception, which
gets converted into a depth change by the time it reaches your
perception. It's what we'll be
trying out here.
In Action
Make a pendulum out of a piece of string and something heavy to use
as a weight, like a bunch of keys. You'll also need
a pair of sunglasses or any shaded material.
Ask a friend to swing the pendulum in front of you in a perpendicular
plane, and make sure it's going exactly in a
straight line, left to right. Now, cover one of your eyes with the
shades (this is easiest if you have old shades and can poke one of
the lenses out). Keep both eyes open! You'll see
that the pendulum now seems to be swinging back and forth as well as
side to side, so that it appears to move in an ellipse. The two of
you will look something like Figure 1.
Figure 1. Matt and Tom use sunglasses and a pendulum made out of a bootlace to test the Pulfrich Effect
Show your friend swinging the pendulum how you see the ellipse, and
ask her to swing the pendulum in the opposite manner to counteract
the illusion. Now the pendulum appears to swing in a straight line,
and the thing that seems odd is not the distance from you, but the
velocity of the pendulum. Because it really is swinging in an
elliptical pattern, it covers perceived distance at an inconsistent
rate. This makes it seem as if the pendulum is making weird
accelerations and decelerations.
How It Works
The classic explanation for the Pulfrich
is this: the shading slows down the processing of the image of the
object in one eye (lower brightness means the neurons are less
stimulated and pass on the signal at a slower rate [Hack #11]);
in effect, the image reaches one eye at a delay compared to when it
reaches the other eye. Because the object is moving, this means the
position of the image on the retina is slightly shifted. The
difference in image perception between the two retinas is used by the
visual system to compute depth [Hack #22]. The slight
displacement of the image on the retina of the shaded eye is
interpreted as an indication of depth, as in Figure 2.
Figure 2. The geometry of the Pulfrich Effect: although the pendulum is, in reality, at point 1, the delay in processing makes it appear to be at point 2 to the shaded eye. When the eyes are taken together, the pendulum therefore appears to be at point 3, at a different length.
This explanation puts the confounding of depth and motion on the
geometry of the situation—the point of confusion lies in the
world, not in the brain.
Taking
recordings of the responses of individual brain cells, Akiyuki Anzai
and colleagues have shown that this isn't the whole
story. The confounding of motion and depth goes deeper than a
mathematical ambiguity that arises from computing real-world
interpretations from the visual images on the retinas.
It seems that most of the neurons in the
primary visual cortex are sensitive to motion and depth in
combination. These neurons are optimally responsive to some
combination of motion and depth; what makes up that optimum
combination can be varying amounts of motion and depth. This means
you when you see something and judge its distance your brain always
also makes a judgment about its velocity, and vice versa. From the
first point in your primary visual cortex where information from the
two eyes is combined (i.e., very early in visual processing), motion
and depth are coupled. You don't get a sense of one
without getting a sense of the other.
This may result from the use of motion parallax to detect depth [Hack #22] . Moving your head is
one of the basic ways of telling how far away something is (you can
see spitting cobras using motion parallax by shifting their heads
from side to side to work out how far to spit). It works even if you
have the use only of one eye.
The joint encoding theory explains why you can get Pulfrich-like
effects in situations with less obvious geometry. If you watch
television snow with one eye shaded, you will see two sheets of dots,
one in front of the other and one moving to the left and one moving
to the right. The reasons for this are complex but rest on the way
our eyes try and match dots in the images for both eyes and use this
matching to calculate depth (stereoscopic vision). Adding a shade to
the image in one eye creates a bias so that instead of perceiving all
the dots at a single average depth we see two sets of skewed
averages, and because depth and motion are jointly encoded, these two
planes move as well (in opposite directions).
In Real Life
The Pulfrich Effect can be used to create
3D effects for television, as long as people are willing to watch
with one eye shaded. It's hard to do since the
motion of the image/camera has to be smooth to create a consistent
illusion of depth, but it has been done.1
