Distance Between Two Points: Great Circle Formula Explained

Figuring out how far apart two points are on a map often comes down to the shape you are allowed to use. For most distances that involve driving or walking, people use a straight-line calculation projected onto a flat map, but that is not always the most accurate way to measure a true geographical distance. In this guide, the exact mathematical and practical limits of distance calculators are explored, specifically the ones that rely on a spherical model of the Earth. You will see why the great-circle formula exists, how different calculation methods produce different results, and why the difference between a flat-map distance and a spherical distance can be significant.

What exactly does the great-circle distance represent?

The great-circle distance is the shortest path between two points on the surface of a sphere, measured along the surface. In practice, when a calculator claims to compute the distance between two cities, it often uses this spherical model because it is much simpler than a true ellipsoidal calculation. For most long-haul flights and maritime routes, this approach is accurate enough, but it introduces subtle errors that a careful user should understand.

For example, if you look at a globe, the line between London and New York looks curved on a flat map, but it is actually a straight line on a sphere. That path is a great circle, and its length is what the formula computes. However, the moment you treat the Earth as a sphere, you are ignoring the fact that it is flattened at the poles and bulging at the equator. That flattening changes the distance by a small but measurable amount.

The mathematical definition is precise: the great-circle distance is the shortest distance along the surface of a sphere between two points. It is computed with Wikipedia (Great-circle distance), which serves as a general reference for the concept and its derivation. This source notes that the shortest distance between two points on a sphere is always a great-circle arc, which is a circle on the sphere’s surface that has the same center as the sphere itself. In practice, this means that the shortest path between two cities on a globe appears as a curve on a standard flat map, but it is a straight line when drawn on a globe. That is the foundation of most distance calculators you use online.

The Coordinately explanation simplifies the concept: the haversine formula is widely used to compute the great-circle distance from latitude and longitude. It is the workhorse formula in many GPS and mapping applications because it represents a computationally efficient way to get a very close approximation of the true distance.

What is the haversine formula and how does it work?

The haversine formula is the most common way to calculate the great-circle distance between two points on a sphere. It works by calculating the central angle between the two points, which is the angle subtended at the center of the Earth by the two points. The formula is designed to be numerically stable even for small distances, which is why it is favored over the spherical law of cosines.

The formula is:

a = sin²(Δφ/2) + cos φ1 ⋅ cos φ2 ⋅ sin²(Δλ/2)

c = 2 ⋅ atan2(√a, √(1−a))

d = R ⋅ c

Here, φ is latitude, λ is longitude, and R is the Earth’s mean radius (6371 km or 3959 mi). The key thing to understand is that the formula computes the central angle c between the two points, and then multiplies it by the Earth’s radius to get the distance. This is a two-step process: first find the angle, then convert it to a distance.

The GeeksforGeeks reference emphasizes that a common mistake is using degrees instead of radians in the trigonometric functions. Since most programming languages and calculators expect radians, you must convert your latitude and longitude from degrees to radians before applying the formula. This is a frequent source of error for people implementing the formula themselves.

For example, if you want to calculate the distance between New York (40.7128° N, 74.0060° W) and London (51.5074° N, 0.1278° W), you first need to convert these coordinates to radians. Then you can plug them into the haversine formula, but only if you have the formula structured correctly. The Underground Mathematics source points out that some references present the formula in different ways, but the underlying mathematics remain the same.

A useful way to think about this is that the haversine formula gives you the straight-line distance through the Earth, but projected onto the surface. The central angle c is the angle between the two points as seen from the Earth’s center. If you know this angle, you can multiply it by the Earth’s circumference and divide by 360 to get the distance along the surface.

The pattern: many tutorials online focus on the formula itself, but they often ignore the practical steps that make it work correctly. You need to be careful with units, and you need to understand that the formula is an approximation of the true distance. If you implement it correctly, you can get results that are within a few percent of the true value.

How does the spherical law of cosines compare to haversine?

The spherical law of cosines is the simpler of the two main formulas for great-circle distance, but it has a significant drawback: it is not numerically stable for very small distances. For example, if two points are close together, the law of cosines can suffer from rounding errors in floating-point arithmetic, which is why the haversine formula is often preferred.

The law of cosines states:

d = R ⋅ arccos(sin φ1 ⋅ sin φ2 + cos φ1 ⋅ cos φ2 ⋅ cos Δλ)

This formula is more compact and easier to read, but it is less reliable for distances less than a few kilometers. In contrast, the haversine formula uses the haversine function (sin²(θ/2)) to avoid these numerical issues. The practical implication is that if you are writing code for a distance calculator, you should choose haversine for robustness.

The Movable Type Scripts reference provides a helpful compromise: it gives a compact version that uses c = 2 * arcsin(min(1, sqrt(a))) and d = R * c, which is essentially the haversine formula but with an arcsin instead of atan2. This can be slightly different in edge cases, but it is still widely used.

The PBSmapping reference clarifies that coordinate inputs for these formulas are usually latitude and longitude in radians, not degrees. This is a common source of error because many people forget to convert their coordinates before using the formulas. If you forget this step, you will get a wildly incorrect answer.

What are the most common mistakes when using distance calculators?

One of the most pervasive errors is the unit mismatch between degrees and radians. Most people think in terms of degrees for latitude and longitude, but the trigonometric functions in most programming languages expect radians. The conversion is simple: multiply degrees by π/180 to get radians. But if you forget this step, your result will be wrong by a factor of about 57.3.

• Using the wrong Earth radius: some calculators use the equatorial radius (6378 km) while others use the mean radius (6371 km). The difference seems small, but it can affect the result by up to 0.1%.
• Forgetting to account for the Earth’s oblateness: the great-circle formula assumes a sphere, but the Earth is an oblate spheroid. For most applications, this is negligible, but for high-precision work, it can be a problem.
• Mixing up latitude and longitude order: some systems use (lat, lon) while others use (lon, lat). If you swap them, you will get a completely different result.

The Transport Geography source warns that the great-circle distance formula assumes points lie on a sphere, not a perfect ellipsoid. This is an important caveat for users who are working with GPS coordinates, which are often measured on the WGS84 ellipsoid. If you use the great-circle formula, you are essentially ignoring the ellipsoidal shape of the Earth.

The Wolfram MathWorld reference additionally explains that the great-circle distance is the length of an arc along a great circle, and it can be computed by direct spherical coordinates or by conversion to Cartesian coordinates. This is another method that is sometimes used, but it is less common because it requires more computation.

Why does the great-circle distance matter for real-world applications?

The practical applications of great-circle distance are everywhere, from airline routing to maritime navigation. When you book a flight, the distance displayed is often the great-circle distance, which is the shortest possible path. However, actual flight paths are often longer because of air currents, no-fly zones, and other operational constraints.

For a logistics company that is planning a delivery route, the great-circle distance gives a lower bound on the travel time. But for a driver, the road distance can be significantly longer because of the road network. This is why it is important to distinguish between great-circle distance and actual travel distance.

The Coordinately guide notes that the haversine formula is a good approximation for most uses, but it is not perfect. For example, if you are measuring the distance between two points that are very close together, the great-circle distance and the straight-line distance on a flat map will be nearly identical. But as the distance grows, the difference becomes more pronounced.

The key difference between the great-circle distance and a flat-map distance is significant. Consider a flight from San Francisco to Beijing: the great-circle distance is roughly 8,700 miles, but if you were to measure the straight-line distance on a flat map and scale it, you would get a distance that is about 1,000 miles longer, depending on the projection used. This is because map projections distort distances, especially at high latitudes.

The implication: if you are working with distances larger than a few hundred miles, you should always use a great-circle calculation rather than a flat-map measurement. This is not just a matter of precision; it is a matter of understanding what the number actually represents.

How can you verify that a distance calculator is accurate?

Verifying a distance calculator is straightforward if you have a reference point. The easiest method is to use two points with known coordinates and compare the calculator’s output to an independent implementation of the formula. For example, you can write a small script that uses the haversine formula and compare the results to an online calculator.

Another method is to use the Movable Type Scripts reference, which provides a compact version of the formula that is often used as a standard for comparison. If your calculator gives a result that differs by more than 0.1%, you should be suspicious.

You can also check the documentation of the calculator to see if it specifies whether it uses the Earth’s mean radius or the equatorial radius. Some calculators let you choose between a spherical and ellipsoidal model, which is the best of both worlds.

The GeeksforGeeks reference notes that the haversine formula uses a and b where a is the square of half the chord length between the points, and b is the square of the half chord length for the longitude difference. This structure makes the formula efficient to compute, which is why it is a staple in navigation systems.

For a more rigorous check, you can use the Wolfram MathWorld reference, which explains that the great-circle distance can be computed using Cartesian coordinates and a dot product. This method is less common, but it provides an independent check on the spherical formula.

What are the practical limitations of distance calculators?

The most important limitation is that the great-circle formula does not account for the actual path you might take. If you are calculating driving distance, you need a different formula that takes into account roads, one-way streets, and speed limits. The great-circle distance is a lower bound on the actual travel distance, but it is rarely the actual distance you would travel.

Another limitation is that the formula assumes a perfect sphere, which is an approximation of the Earth’s true shape. The Earth is actually an oblate spheroid, which means it is wider at the equator and narrower at the poles. This discrepancy can cause errors of up to 0.3% for long distances.

The Transport Geography source emphasizes that the great-circle formula is best used for air and sea navigation, where you have the freedom to travel in any direction. On land, you are constrained by the road network, so the great-circle distance can be meaningless for planning a road trip.

If you are using a calculator that provides both great-circle and ellipsoidal distances, you can see the difference for yourself. For short distances, the difference is negligible. For distances of a few thousand kilometers, the difference becomes noticeable, and for distances on the order of 10,000 km, the difference can be as large as 50 km.

The catch: if you are comparing these two methods, make sure you are comparing the same radius value. Some calculators use the mean radius (6371 km), while others use the equatorial radius (6378 km). This small difference can lead to a discrepancy of about 0.1%, which is acceptable for most purposes but can be confusing.

What are the key takeaways for using these formulas correctly?

To use great-circle distance formulas correctly, you need to understand the context in which they apply. These formulas give you the shortest distance between two points on a sphere, which is a useful approximation for most real-world scenarios. However, you must be aware of the assumptions and limitations.

• Always convert latitude and longitude from degrees to radians before applying the formula.
• Use the Earth’s mean radius (6371 km) unless you have a reason to use a different value.
• For distances less than a few kilometers, use the haversine formula to avoid numerical errors.
• For high-precision work, use an ellipsoidal model instead of a spherical one.
• Remember that the great-circle distance is a straight-line distance along the surface, not the actual travel distance.

The Underground Mathematics reference clarifies that some sources use a different notation for the formula, but the principle is the same. The central angle c is the distance between the two points in angular measure, and multiplying it by the Earth’s radius converts it to a linear distance.

The PBSmapping reference additionally explains that the haversine formula is a special case of the more general great-circle formula, and it is particularly well-suited for short distances because it avoids rounding errors that can occur with other methods.

The consequence of ignoring these steps is a world of inaccurate results that can lead to poor planning. For example, if you are calculating the distance for a flight and you use a flat-map measurement instead of a great-circle measurement, you could overestimate the distance by as much as 10%.

From a mathematical perspective, the great-circle distance is a beautiful example of spherical geometry in action, but from a practical perspective, it is a tool that must be used with an awareness of its limitations.

Wolfram MathWorld

In practice, the haversine formula is a good balance between accuracy and simplicity, making it the go-to choice for most geographic information systems.

Movable Type Scripts

For a final summary, the most trustworthy approach is to understand the formula you are using, and to verify your results against a known reference. Whether you are a developer implementing a distance calculator or a user who wants to double-check a measurement, a solid grasp of these concepts will serve you well. The next time you see a distance on a map, remember that it is not just a number; it is the result of a specific mathematical operation.

Related reading: How Many Cups in a Quart? – Quick Conversion Guide · 72 Hours From Now: Exact Time, Date, and Calculator Guide

Frequently asked questions