The Physics Behind the Magical Parallax Effect Running Your AR Apps

March 20, 2019 Off By HotelSalesCareers

There's something sort of cool in the next version of Apple's iOS. It's called ARKit—basically, it's a part of Apple's developer package to help programmers create awesome augmented reality apps. Like, maybe a program that adds dancing hotdogs to your screen so that they look like they are there in real life. Or better yet, something useful—like an app that measures distances by just looking at stuff through your phone's camera.

But how does this work? What magic does Apple use so that your phone can turn a 2-D image into something that looks like it is there in real life? The answer is parallax.

Let's start with a super simple demo to demonstrate the effect. Here's what you do: Take your arm and hold it out in front of you with your thumb sticking up. Now close your left eye and look at your thumb. In particular, look at some object that is past your thumb (something in the room or something outside—it doesn't matter). Now open your left eye and close your right eye. Notice the apparent motion of your thumb with respect to background objects. It looks as though your thumb is moving. Now switch your viewing eye back and forth—left, right, left, right. Cool winky face, dude.

The moving thumb is an example of parallax. It's the apparent motion of an object with respect to background objects when the viewing point moves. The closer the object is to an eye (or a camera), the more it appears to move. In the case of your eyes, it's as though your view moves from the left to the right eye. You could also do this with an actual camera.

But here is the cool and useful part. If you know the actual distance between two viewing points and the angular change in position of an object, you can calculate the distance to the object. Parallax isn't just a cool party trick, it can also be used to find real stuff. This calculation isn't even that difficult. If you measure the angular displacement of the object and the horizontal shift, then the distance to the object would be:

Of course, the angle should be in units of radians in this cases—but your phone can determine this value by comparing two views from the camera (and using some object recognition algorithm). What about the shift distance? Yes, your phone can do this too. Using sensors like the accelerometer, it can calculate the distance the phone moves from one vantage point to another. So there you have it. That is how the ARKit works—well, probably.

Parallax and the Heliocentric Model

As long as I am talking about parallax, I should include the most important and interesting example from astronomy: stellar parallax.

An early model of the solar system, the geocentric model, held that the Earth was in the center of the solar system—the other planets and the sun revolved around it. At first glance, this model seems to make sense. We can see that the planets do indeed move relative to the background stars. And it doesn't really feel like the Earth is moving, so why not put the Earth in the center?

After collecting more data on the motions of the planets, scientists started leaning toward a heliocentric model of the solar system, with the planets instead orbiting around the sun. But there was one big stumbling block to the adoption of this model—the lack of stellar parallax.

Just imagine that you are on the Earth (because you probably are). In the month of December you are on one side of the sun. Six months later, you are on the other side. The view from these two locations are like a ginormous pair of eyes looking at the stars from different view points. By knowing the size of the Earth's orbit and the angular shift of the stars, you can use parallax to determine the distance to these stars. This is called stellar parallax.

However, there was a big problem. Early astronomers attempted to measure the angular shift of stars as the Earth went around the sun, but they couldn't see any shift. Maybe there was no stellar parallax because the Earth wasn't moving around the sun. Maybe the geocentric model was the better model.

The answer lies in the distance to the stars. If the Earth orbits the sun (and it does), then we can work backwards and calculate the angular motion of the nearest star. Alpha Centauri is about 4 light years away (or about 250 thousand times further than the Earth is from the sun). Using the parallax formula above, this should give an angular shift of 1.6 x 10^-5 radians or just 3.3 arc-seconds. This tiny angle was just too small to detect with the instruments of the time. But it is there. Traditionally, the angular shift is reported as half of the total angle (just a note).

What if an object was a at a distance such that it had a half-angle shift of 1 arc-second? This would have the distance of 1 parsec. Yes, a parsec is a unit of distance, not time. The "par" is for parallax and the "sec" is for angle.

Side note: Yes, that means Han Solo was wrong. The Millenium Falcon couldn't have made the Kessel Run in under 12 parsecs—that's not even a time, even though it sounds cool. Really, if Star Wars had a science advisor, they should have recommended some other time unit. How about "megasec"? Or even better you could use "quantsec"—which doesn't really mean anything but sounds cool. Oh, don't worry—I still love Star Wars and this Kessel Run thing doesn't even bother me.

The Physics Behind the Magical Parallax Effect Running Your AR Apps

Parallax and the Heliocentric Model

Related Stories

Related Video

AR, VR, MR: Making Sense of Magic Leap and the Future of Reality