The Earth is not a sphere: datums, heights and distance for GeoAI

School teaches that the Earth is a sphere. It is close, and for many purposes the approximation is fine. But the moment you measure distances across a province, fly a drone with centimetre accuracy or overlay a 3D model on the real world in mixed reality, the details start to matter. This article explains the shape of the Earth as mapmakers see it, and the practical consequences for GeoAI, 3D and VR.

A flattened planet

The Earth spins, and the spin makes it bulge at the equator and flatten at the poles. The equatorial radius is about 6,378 kilometres, and the polar radius is about 6,357 kilometres, a difference of roughly 21 kilometres. The best simple shape for this is an ellipsoid, a sphere squashed from the top and bottom. The amount of flattening is about one part in 298.

Surveyors define a reference ellipsoid, with an exact size and shape, as the mathematical model of the Earth. Latitude and longitude are measured on that ellipsoid.

What a datum is

A reference ellipsoid is not enough. You also have to fix it to the real Earth: where its centre is, and how it is oriented. That choice is called a geodetic datum. Different datums are different choices.

• WGS 84 is a global datum, centred on the Earth’s centre of mass. GPS uses it, and most web maps assume it.
• Many countries and regions have local datums, chosen to fit their own territory best, and many old maps and survey records use them.
• Modern national systems are increasingly tied to global ones, but older data may still carry the local datum.

Why does this matter? Because the same latitude and longitude in two different datums can describe places tens or even hundreds of metres apart. If you combine data from two sources without converting, features will be shifted against each other, and a model trained on one may misalign with the other.

Three kinds of height

Height is more complicated than it seems.

• Ellipsoidal height is the height above the reference ellipsoid. It is what a GPS receiver computes directly.
• The geoid is a surface that follows mean sea level and extends under the continents, shaped by gravity. It is lumpy, rising and falling by up to about a hundred metres relative to the ellipsoid across the globe.
• Orthometric height is the height above the geoid, which is what we normally mean by “height above sea level”, and what maps and elevation models usually show.

The difference between ellipsoidal and orthometric height at a place is called the geoid undulation, and it can be many metres. A GPS reading that says you are 1,400 metres up may correspond to a very different elevation on a map, unless a geoid model is applied. Mixing the two is a classic source of errors in terrain work, flood models and drone mapping.

Why you cannot use Pythagoras on latitude and longitude

A degree of latitude is nearly the same length everywhere, about 111 kilometres. A degree of longitude is not. It shrinks as you move away from the equator, in proportion to the cosine of the latitude. At about 27.7 degrees north, near Kathmandu, one degree of longitude is only about 98.6 kilometres.

If you treat degrees as flat x and y values, then you will overestimate east-west distances at every latitude away from the equator. Between Kathmandu and Pokhara, the real distance, along the surface of the globe, is about 142 kilometres. Multiplying the difference in degrees by 111 on both axes gives about 158 kilometres, an error of 11 percent.

Distance on a sphere: the haversine

For a good approximation, treat the Earth as a sphere with a mean radius of about 6,371 kilometres and use the haversine formula. It computes the great-circle distance, the shortest path over the surface, from two latitude and longitude pairs. It is accurate to roughly half a percent, which is plenty for many tasks. For survey-grade work, use a geodesic algorithm on the ellipsoid, available in common libraries.

1. 01Convert latitudes and longitudes to radians.
2. 02Compute the differences in latitude and longitude.
3. 03Apply the haversine formula, which combines sines of half the differences with the cosines of the two latitudes.
4. 04Multiply the resulting angle by the Earth’s radius.

How many decimals are enough?

Coordinates in decimal degrees carry more precision than they appear. The fifth decimal place of a degree is roughly 1.1 metres, and the seventh is about one centimetre. Storing coordinates with six or seven decimals is plenty for almost every task. Reporting a dozen decimals suggests a precision that the measurement never had.

What this means for 3D, VR and mixed reality

• Floating-point precision. Computers store numbers with limited digits. A 32-bit number near five million metres can resolve only about half a metre, so objects far from the scene’s origin visibly jitter. The fix is to use a local origin near the area of interest, and store global coordinates separately in higher precision.
• Alignment with the real world. Mixed reality that must line up with real buildings needs consistent datums and reliable positioning, and may need extra reference points.
• Scale. A model built in one coordinate system and imported into another can change size or orientation if units and axes differ.
• Terrain. Elevation models and GPS heights must agree on which kind of height they use.

A checklist for any project

1. 01Record the datum and coordinate system of every dataset you receive.
2. 02Convert everything to one system before combining.
3. 03Know which height each dataset uses, and apply a geoid model if you need to compare.
4. 04Use a projected system suited to your region for measuring, and a geodesic or haversine formula for long distances in degrees.
5. 05Test with two points whose real distance you know.

Frequently asked questions

Is WGS 84 the same as the system used by Google Maps?

They use the same datum, but display maps in a projected system called Web Mercator, which distorts areas near the poles. Stored coordinates are usually WGS 84 latitude and longitude.

Why does my GPS height not match the elevation on my map?

GPS gives height above the ellipsoid. Maps show height above sea level, which follows the geoid. The two differ by the geoid undulation, which can be many metres.

When should I use a geodesic instead of the haversine?

When you need accuracy better than a few tenths of a percent, for example in surveying, or over very long distances. For general analysis, the haversine is fine.

Does the datum matter for satellite imagery?

Yes. Imagery is placed on the ground using a datum. Mixing images or vectors from different datums without converting will shift features against each other.

The Earth’s shape is not an obscure detail for surveyors alone. It affects any project in which data from different sources must agree. Know your datum, know your height, and test your distances, and a whole family of subtle errors disappears.