How cameras capture 3D: optics, parallax and photogrammetry

Drone maps, 3D models of heritage sites and photorealistic virtual tours all begin with the same humble object: a camera. A photograph flattens a three-dimensional world into two dimensions. Photogrammetry is the craft of reversing that, recovering shape and measurement from many overlapping images. It is the engine behind much of the 3D content used in GeoAI, virtual reality and mixed reality, and knowing how it works explains both its power and its limits.

The pinhole camera model

The simplest way to understand a camera is a dark box with a tiny hole. Light from each point of the scene passes through the hole and lands on a single point of the sensor, forming an upside-down image. Real lenses gather more light, but the geometry is the same: each pixel corresponds to a ray of light coming from a particular direction.

Two numbers describe this camera. The focal length is the distance from the lens to the sensor. The sensor size sets how much of the view is captured. Together they give the field of view. A larger focal length, a telephoto lens, sees a narrow slice of the world, and a short one sees a wide scene.

• Field of view = 2 × arctan (sensor width ÷ (2 × focal length)).
• A one-inch sensor, 13.2 millimetres wide, behind an 8.8 millimetre lens sees about 74 degrees across.

Ground sample distance from the air

When a drone or aircraft photographs the ground, how large is one pixel on the ground? The answer follows from similar triangles: the ground sample distance equals the flying height multiplied by the pixel pitch, divided by the focal length. Pixel pitch is the size of one pixel on the sensor.

• Sensor width 13.2 mm, 5,472 pixels across: pixel pitch is about 2.41 micrometres.
• Focal length 8.8 mm, flying height 100 m: GSD = 100 × 0.00000241 ÷ 0.0088, which is about 2.7 centimetres per pixel.
• Fly twice as high and the pixels become twice as large, but each photo covers four times the area.

This is the formula that lets you plan a flight: choose the detail you need, and it tells you the altitude.

Depth from two views: parallax

Hold a finger in front of your face and close each eye in turn. The finger seems to jump sideways against the background, a lot more than a distant tree does. That shift is parallax, and its size tells you the distance. Two photographs taken from different positions work the same way. The distance between the two camera positions is called the baseline.

• Nearby objects shift a lot between the two images, and distant ones barely move.
• With a known baseline and focal length, the shift (the disparity) gives depth by triangulation.
• A longer baseline gives better depth accuracy, but makes it harder to match points between images.

Structure from motion: many photos, one model

Modern photogrammetry applies that principle to hundreds of photos at once. A typical pipeline has these steps.

1. 01Find features: the software detects thousands of distinctive points in every image, such as corners and textures.
2. 02Match them: it finds the same points in overlapping images.
3. 03Solve the geometry: it works out where each camera was and where each point lies, adjusting everything together until the errors are as small as possible. This step is called bundle adjustment.
4. 04Densify: it computes depth for most pixels, producing a dense point cloud.
5. 05Build a mesh and a texture: the points become a surface covered with image colour.
6. 06Produce outputs: an orthomosaic (a map-accurate image), an elevation model or a 3D model.

Planning the capture

• Overlap. Mapping flights usually need about 70 to 80 percent overlap along the flight direction and 60 to 70 percent between lines, so that every point appears in several images.
• Consistent light. Overcast days give soft, even light. Strong shadows that move during the flight make matching harder.
• Sharp images. Fast shutter speeds avoid motion blur, and a steady drone helps.
• Control points. Marked points on the ground, surveyed with accurate GNSS, tie the model to real-world coordinates and improve accuracy.
• Calibration. Lenses distort images slightly, and software estimates and corrects this.

What goes wrong

• Surfaces without texture, such as calm water, fresh snow and plain walls, give the software nothing to match.
• Shiny or transparent surfaces, such as glass and polished metal, change appearance with the viewpoint.
• Moving objects, such as cars, people and swaying trees, appear in different places in different photos.
• Thin structures, such as wires and railings, are often lost or smeared.
• Too little overlap leaves holes and weak geometry.
• A rolling shutter, found in many cameras, distorts fast-moving scenes.

Laser scanning as an alternative

LiDAR measures distance directly. It sends a pulse of laser light and times its return. Distance is the speed of light multiplied by the travel time, divided by two, because the light goes out and back. Scanners from a drone, aircraft or tripod record millions of points per second, and the pulses can pass between leaves to reach the ground. It works well in forests, in poor light and on plain surfaces, but the equipment is more expensive, and it does not record colour on its own. Many projects combine both.

From capture to VR and mixed reality

• Raw models contain millions of polygons. Headsets and phones have limited power, so models must be simplified.
• Levels of detail show a detailed version up close and a lighter one far away.
• Textures are compressed, because memory is limited.
• Real-world scale and orientation must be correct, or the scene will not fit the room or the site.
• Accuracy depends on ground sample distance and geometry. A model good enough for a tour may not be good enough to measure a crack.

Frequently asked questions

How accurate is drone photogrammetry?

With good overlap, calibration and surveyed control points, results of a few centimetres are achievable. Without control points, the absolute position may be off by metres, though relative measurements within the model can still be good.

How many photos do I need?

It depends on the area and the detail. A small site may need a few hundred. What matters more is the overlap and the consistent coverage.

Can I use photos from a phone?

Yes, for small objects and scenes, and many people do. Keep the camera settings fixed, move steadily, and cover the object from many angles.

Photogrammetry or LiDAR?

Photogrammetry is cheaper and gives colour, and suits open, textured scenes. LiDAR suits vegetation, plain surfaces and low light. If you can, use both.

Photogrammetry feels like magic because the software does so much for you. But it rests on school-level geometry and the physics of light, and understanding it lets you plan better flights, spot bad models quickly and know what accuracy to promise.