Fun With Science / Globe Deconstruction / Q12 · page 55
Q12, p. 55, argued at p. 202 — refraction is a ground-level problem, a high sun is the regime where it behaves, and many observers is the control for it.
Nobody proved that sunlight travels straight through air, because it does not: air bends light, and the bend has been measured since Tycho Brahe. The premise is conceded in full. The question’s force comes from a switch of regime. Refraction is a large, unruly, unpredictable effect along the ground, which is where the surveyor’s constant on p. 202 comes from, and a small, stable, routinely-corrected one for a sun high in the sky, which is where shadow angles are measured. For a sun 45° up it is about one arcminute — a sixtieth of a degree; an arcsecond is a sixtieth of that — and the argument needed degrees.
Refraction is real: conceded in fullNear the ground: large and erratic, concededTwo sticks prove sphericity: overclaimedIt reaches noon shadow angles: 7.5″ against 25,596″Many observers a weakness: inverted — it is the control
Where this lands
The claim: nobody proved sunlight travels straight, so multi-station shadow-angle measurements cannot rule out a flat Earth. The verdict: refuted. The premise is true and the conclusion does not follow from it. Across the Alexandria–Syene baseline refraction is worth 7.5 arcseconds against a signal of 25,596. Where the flat model actually needs bent light — at the Arctic Circle, set against the measured length of a degree of latitude — it needs 17.01° of bending and the atmosphere supplies 0.0375°: short by a factor of 454 on standard refraction, 384 on the coldest polar air.
Bending that large cannot be uniform, and non-uniform bending changes the Sun’s shape. The flat model needs a mid-afternoon Sun squashed by a quarter. The Sun is round.
Three things carry the page. One: refraction is overwhelmingly a horizontal problem. A ray running along the ground never leaves the disturbed surface layer, and its refraction coefficient k — the ray’s curvature as a fraction of the Earth’s — has been measured swinging from −4 to +16 against a textbook 0.13. A ray from a high sun crosses that layer once, steeply, and picks up about seven hundred times less bending. Two: above about ten degrees of elevation the atmosphere’s profile cancels out of the bend exactly, leaving only the air density where the observer stands — a barometer and a thermometer reading. Across the whole envelope of terrestrial weather, refraction at 45° stays inside a hundredth of a degree of one arcminute. Three: ten observers at ten latitudes do not compound the problem. Their errors are uncorrelated and average down while the signal grows with baseline, and the scatter between them measures the residual without anyone assuming anything about the air. The page answers the noon case, which is where shadow-angle work is done; where that choice could matter is set out in section 8.
The question card at p. 55 is the short form. The claim is set up at p. 34, in a list of what globe defenders offer as proof: “Sun elevation angle measurements using shadows across latitude lines up with mathematical precision with the globe.” It is taken up at pp. 35–36, where Eratosthenes heads four proofs to be dismantled, and where two assumptions are named — that sunlight is effectively parallel on arrival, and that the Earth is a sphere — of which he writes, “He did not verify curvature before this experiment was conducted.” Page 36 then hands off explicitly: the inconsistent-sun-height objection rests on an unproven assumption “which is explained on page 202.”
Page 202 is the argument. Headed “What about the shadow-angle measurements with the Sun that ‘match’ globe predictions?”, it names the coordinated experiments — “10 or more participants at varying latitudes” — and their “one critical assumption”: “that a scientist has proven that the Sun’s light is completely straight over long distances.” The mechanism named is terrestrial refraction: surveyors and geodesists cannot assume their laser levels or sightlines are straight, and carry a constant to account for it. Beside the text is a table headed standard atmospheric refraction, k = 0.13, captioned: “This table proves we cannot assume sunlight travels in straight lines over long distances through the atmosphere.” The right-hand column below recomputes it from the surveyor’s formula, Δh = k D²/2R: the apparent lift of a target at distance D on an Earth of radius R.
| Distance | Atmospheric shift, as printed | …in metric | k D²/2R recomputed |
|---|---|---|---|
| 0.62 mi | 1.1 cm | 1 km | 1.10 cm |
| 3.1 mi | 27 cm | 5 km | 27.4 cm |
| 6.2 mi | 1.1 m | 10 km | 1.10 m |
| 12.4 mi | 4.4 m | 20 km | 4.39 m |
| 31.1 mi | 27.5 m | 50 km | 27.6 m |
| 62.1 mi | 110.1 m | 100 km | 110.0 m |
The table is correct. Two observations about it, one trivial and one not.
The trivial one: the numbers are computed with k = 0.14 and the polar radius, not the k = 0.13 printed in the heading. The recomputed column above uses his figures rather than his caption, which is why it agrees to the last digit; at 0.13 the rows would read 1.0 cm, 25 cm, 1.02 m, 4.06 m, 25.6 m, 101.9 m. Nothing turns on it — 0.14 is a perfectly ordinary default, the one Topcon ships and WSDOT specifies — but the two do not match.
Read the bottom row for what it says, too. 110 m of displacement over 100 km sounds enormous, but expressed as the thing a shadow-angle measurement cares about, a direction, it is k D/2R = 3.8 arcminutes — for a ray which spent a hundred kilometres inside the surface layer. The whole atmosphere, crossed once by a sun 45° up, supplies about one. Run his own law far enough to matter and it says how far: to accumulate the 17° the flat model is short at the Arctic Circle in section 7, a ray would have to graze the ground for about 27,000 km — two-thirds of the way around the world. Apparent shift throughout, kD/2R, which is what a theodolite reads and the right analogue for an apparent sun elevation; the total bending of the ray is twice it.
The usual textbook presentation of Eratosthenes states the parallel-ray assumption in passing, puts no number on it, and does not mention refraction; asking for the number is fair. So the question can only be answered by measuring the bend, in the regime where the measurement is made, and showing what size of conclusion it can and cannot overturn.
Neither of the two things p. 36 says Eratosthenes “assumed” — parallel sunlight and a spherical Earth — was an assumption. Both were prior results, reached by observations with nothing to do with shadow lengths.
The Sun’s distance had been measured, by a method that says nothing about the Earth’s shape. Aristarchus of Samos (c. 310–230 BC), a generation before Eratosthenes, saw that when the Moon is exactly half-lit the Sun–Moon–Earth angle at the Moon is a right angle, so measuring the remaining angle from Earth gives the ratio of the two distances. He measured 87° and concluded that the Sun’s distance is “greater than 18 times, but less than 20 times, the distance of the Moon,” with a diameter between 19/3 and 43/6 that of the Earth. Geometry plus one angle; no Earth model enters it. He was badly wrong — the true dichotomy angle is about 89.85°, which no naked eye can tell from 90°, so his answer came out roughly eighteen times too small — and it made no difference, because eighteen times too small was still an order of magnitude more than the measurement needed.
| Sun at… | Ray convergence across the 843 km baseline | As a share of the 25,596″ signal |
|---|---|---|
| the true distance | 1.16″ | 0.005% |
| Aristarchus’ figure, 18–20 lunar distances | about 21″ | 0.08% |
| The 21″ follows his own chain — 18–20 times a lunar distance he put at roughly 70 Earth radii, so about 1,260–1,400 Earth radii. Using the modern lunar distance instead gives 23″. Either serves. | ||
| …and how near it would have to be before the assumption bit: | ||
| under 1.8 lunar distances | 256″ | 1% |
So the parallel-ray premise was a measured lower bound, an order of magnitude clear of where it would have started to matter, produced by someone doing something else entirely.
And the Earth’s shape was a prior conclusion too. Aristotle, around 350 BC, gave two observational arguments in On the Heavens: the Earth’s shadow on the Moon during a lunar eclipse is round every time, at every orientation, which a disc cannot manage except face-on; and southern constellations stand higher in the sky for southern observers than northern ones. Neither involves a shadow length or a sun elevation, so neither is circular with respect to what Eratosthenes went on to measure. They are inferences from repeated observation, not measurements with error bars, and section 7 concedes that a two-station shadow measurement cannot by itself decide between a far sun on a globe and a near sun over a flat plane. The narrow claim here is the one p. 36 disputes: sphericity was arrived at independently, from evidence of a different kind.
Hirt and colleagues put two pairs of motorised total stations on a surveyed 800 m baseline, 1.8 m above grass, and took reciprocal vertical angles once a minute for 33 hours across five days. The refraction coefficient k ran from −4 to +16 on sunny days, against the textbook 0.13, fluctuating by 1 to 1.5 on ten- to thirty-minute timescales. For a grazing sightline the standard number can be wrong by two orders of magnitude, and in both directions.
Engineers who need light to travel straight over that kind of distance take the air out. SLAC’s two-mile linear accelerator was aligned through a 24-inch evacuated pipe at roughly 10–25 torr — a thirtieth of sea-level pressure or less — explicitly “to avoid distortion and translation of the image due to refraction of the light in the atmosphere.” Asked whether a 250 m system would need vacuum, Herrmannsfeldt answered: “you will need some kind of vacuum, yes… 250 m is too long for reliable work.”
A physicist who will not trust air over a 250 m horizontal sightline is saying the effect is not negligible in general. It is negligible for a particular geometry, and the reason the two regimes diverge so violently falls out of the same formalism that governs both. The bending a layer contributes is Δn · tan z, where n is the refractive index of air and z is the ray’s angle from the vertical, the zenith angle. Take a 2 m surface layer with 10 K of temperature drop across it: Δn = (n−1)·ΔT/T = 9.63 × 10⁻⁶. A ray at 60° elevation crosses that layer once and picks up 9.63 × 10⁻⁶ × tan 30° = 1.15 arcseconds. A horizontal sightline never leaves it: the ray curvature κ = 4.81 × 10⁻⁶ per metre acts over the whole 800 m, giving 790 arcseconds of direction change and about 1.5 m of apparent target rise.
Same air, same gradient, and a factor of about seven hundred in the bending — purely from how long the ray stays inside the disturbed layer.
That 10 K over 2 m corresponds to k = 30.7, beyond Hirt’s observed maximum, so it is an extreme rather than a typical case. Using 2 K gives k = 6.1, comfortably inside his measured range, and 0.23 arcseconds at 60° elevation. The conclusion does not depend on which is chosen. Our flat-surface test page handles the near-horizon case in full, including the surveyor’s k ≈ 0.13–0.17 and the seven-sixths-Earth rule; that page, not this one, is where the grazing-incidence argument properly lives.
Two things a surveyor would add, worked through in the method notes at the end of this page: the 0.13 was never a near-ground number in the first place — it belongs to the intermediate atmosphere from about 20–30 m up, where it is well behaved — and survey practice keeps sight lines short enough that even Hirt’s wildest excursions are worth millimetres. In the discipline that supplies the constant, refraction is the small term, a seventh of the curvature correction it is always paired with. Either way it is a value for short sight lines near the ground. A sunbeam at noon is neither short nor near the ground.
The phrase invites you to picture hundreds of miles of accumulated unknown. The physics deletes that unknown, exactly.
Model the atmosphere as horizontal layers of differing density and apply Snell’s law at each interface. The refracted angle leaving one layer is the incident angle entering the next, so the product n · sin z is identical at every point along the ray. Every intermediate layer cancels. Only the endpoints survive. A prepared reader will object that the derivation stacks flat layers, which is the shape under dispute. On concentric shells the invariant becomes Bouguer’s n r sin z = constant, and above roughly 10–15° of elevation the two agree to well inside the arcsecond figures used here. The plane-parallel form is the simpler one to show; nothing on this page depends on which is used.
The total bending does not depend on the density profile, the temperature profile, the scale height, the number of layers, or how many miles of air the ray crossed.
It depends on the refractive index of air where the observer is standing, and on the zenith angle. That is the whole of it, and it reduces to r = (n − 1)·tan z, which Young’s treatment states is good to better than one part in ten thousand while r stays under about half a degree — covering everything above roughly 10° of elevation, which is to say everything a noon shadow measurement ever uses.
The sign is known too. Where density falls monotonically upward — the ordinary case for a high sun on a clear day — the ray curves toward the denser air beneath it, the source appears higher than its true direction, and the shadow is shortened. Surface inversions and horizontal gradients, the conditions that produce mirages, belong to the ground-level regime of section 3. For a high sun, refraction is a correction with a known sign, not a scrambler.
And the index itself is not an astronomical quantity. The Ciddor dispersion relation gives n − 1 = 2.7715 × 10⁻⁴ at 589 nm for standard air at 15 °C and 101.325 kPa. NIST serves it through its Engineering Metrology Toolbox, because laser interferometric measurement of gauge blocks cannot be done without correcting for the optical path in air. It exists so that machine shops can agree on what a millimetre is.
This is why nobody mentions refraction above ten degrees. Not because it vanishes — shadow-angle work that wanted arcsecond accuracy would correct for it — but because it gets smaller, by a factor of about thirty-five from the horizon to 45°, and because the kind of uncertainty changes. Near the ground, what dominates is the shape of the vertical gradient in the first few metres of air, the profile, which nobody can model; it is what Hirt was measuring when k came out between −4 and +16. Above roughly ten degrees the invariant removes the profile from the problem entirely, and what is left depends only on the density of the air at the observer: a barometer reading and a thermometer reading, which every station has.
The near-horizon regime has an error nobody can compute. The high-sun regime has an error anyone can look up.
Here is the distribution, which is the thing p. 202 gestures at without giving. The middle column is ordinary conditions. The two beside it are the envelope of terrestrial weather: 840 hPa at +30 °C, and the record sea-level pressure of 1083.8 hPa recorded at Agata, Siberia in 1968, at −50 °C. Those extremes are bounds, not a forecast — no station experiences either, and certainly not both. The envelope captures bulk density, which under the invariant is the whole story for a high sun; near the horizon, where profile anomalies put the real bending outside any pressure-and-temperature envelope, section 3 applies instead.
| Sun elevation | Hot and thin 840 hPa, +30 °C | Ordinary 1010 hPa, +10 °C | Cold and dense 1083.8 hPa, −50 °C | Full spread |
|---|---|---|---|---|
| horizon | 26.76′ | 34.45′ | 46.92′ | 20.16′ |
| 5° | 7.67′ | 9.88′ | 13.45′ | 5.78′ |
| 10° | 4.19′ | 5.39′ | 7.34′ | 3.15′ |
| 20° | 2.10′ | 2.70′ | 3.68′ | 1.58′ |
| 30° | 1.33′ | 1.72′ | 2.34′ | 1.00′ |
| 45° | 0.77′ | 0.99′ | 1.35′ | 0.58′ |
| 60° | 0.45′ | 0.57′ | 0.78′ | 0.34′ |
| 75° | 0.21′ | 0.27′ | 0.36′ | 0.16′ |
Two independent routes agree on the middle column. The metrology-bench value fed into r = (n−1)·tan z gives 57.2″ at 45° elevation for 15 °C air and 60.3″ for 0 °C; Bennett’s 1982 navigation formula, fitted to sky observations for sextant correction rather than derived, gives 59.7″. The residual is mostly the few degrees of temperature separating the two conventions. Bennett is quoted here with its pressure–temperature factor set to unity, corresponding to about 1010 hPa and 10 °C; the envelope columns apply that factor at its extremes. The whole table is one formula and two numbers, so a reader can rebuild it.
Page 202 treats the coordinated multi-latitude experiments as the thing that makes the proof vulnerable. It is the opposite: that design is the standard way of controlling for exactly the error being alleged, for four reasons that can each be checked.
That fourth one deserves a number rather than a rhetorical question, because there is a genuine latitude-correlated systematic.
A second systematic, which averaging also does not touch: if every observer uses the same flawed procedure, the flaw survives any number of observers. That is a real limitation of the design, and it argues for observers who are not coordinating on method — which is what the eighteenth-century arcs in section 7 supply, surveyed by rival teams to settle a different question.
The arithmetic, with inputs so it can be redone. Alexandria sits at 31.20° N and Syene at 24.09° N, with the obliquity — the tilt of the Earth’s axis — about 23.72° in 240 BC. Noon solstice elevation is 82.52° at the first and 89.63° at the second. Bennett gives 7.79″ and 0.31″. Both stations are lifted; only the difference touches the answer, and the difference is 7.5 arcseconds against a signal of 7.11° — 25,596 arcseconds. That is a perturbation of 0.029 per cent, about 12 km on a 40,000 km circumference.
Set it against the measurement’s own error budget. Eratosthenes reported 252,000 stadia, and the stadion is known only to lie between about 155 and 160 m — a range of −2.4% to +0.8% against the true figure.
Refraction is about seventy times smaller than the uncertainty the ancient measurement already carried.
The parallel-ray assumption gets the same treatment. The astronomical unit is fixed at 149,597,870,700 m by IAU 2012 Resolution B2, because ranging measures it directly by time of flight. Across the 843 km between the two sites — the true separation, against the 787.5 km his own 5,000-stadia figure implies — the direction to the Sun differs by 1.16 arcseconds. So the two optical assumptions Q12 targets are worth 7.5″ and 1.16″ against 25,596″.
This is the part of the objection with real force. Take his own datum — 5,000 stadia, which at 157.5 m to the stadion is 787.5 km — and 7.2° of angle. On a globe the circumference is 787.5 × 50 = 39,375 km. On a flat plane with a nearby sun, treating Syene as the subsolar point — the spot directly beneath the sun — the sun’s height is 6,234 km. Both fits are exact. Two observations, two models, one dataset, nothing to choose between them. Anyone presenting two sticks as proof of sphericity has claimed more than the data supports.
What resolves it is a third station and a fourth — section 6’s design. The discriminator is unusually clean, because the flat model’s one free parameter cancels out of it. On a plane with the sun at height H, ground distance to the subsolar point is x = H tan φ, so ground distance per degree of observed elevation is H sec²φ (π/180). H is an overall scale and divides out of every ratio.
And that quantity was measured, repeatedly, by people arguing about something else: the seventeenth- and eighteenth-century arcs were run to settle whether the Earth is oblate as Newton predicted or prolate as Cassini did, a dispute in which neither side had any stake in this one.
| Survey | Latitude | Measured degree | Ratio to equatorial | Flat model requires |
|---|---|---|---|---|
| Bouguer & La Condamine, Peru | ≈0° | 110,577 m | 1.0000 | 1.000 |
| Picard, Paris–Amiens | ≈49.4° | 111,212 m | 1.0057 | 2.361 |
| Maupertuis, Torne valley | ≈66.3° | 111,918 m | 1.0121 | 6.190 |
Converted at 1.949036 m to the toise du Pérou. Maupertuis’ Lapland figure is about 0.36% larger than the modern value, and was contested at the time. An error of 0.36% cannot rescue a prediction wrong by a factor of 6.19.
That derivation assumes rays travel straight. A reader can escape it by proposing that the air bends sunlight enough to fake the observations. This is the escape route the whole question is really about, so here is what it costs. The ground distance is taken from the three measured arcs, not from the modern ellipsoid, so that nothing in the calculation assumes the conclusion.
Calibrate the flat model’s sun height on the measured equatorial degree, where the two models agree best: H = 110,577 m ÷ (π/180) = 6,336 km. The ground distance from the equator to 66.3°, trapezoidal across the three surveys above, comes to 7,364 km. The flat model then predicts an equinox noon elevation there of arctan(6336 ÷ 7364) = 40.71°. Every observer there measures 23.70°.
The gap is 17.01 degrees. The atmosphere at that elevation supplies 2.25 arcminutes — 0.0375°. It is short by a factor of 454 on standard refraction; on the coldest polar air of section 6, 2.66′, the factor is 384.
Grant the flat model the largest refraction the atmosphere produces anywhere on Earth — the 34.5 arcminutes at the astronomical horizon, 0.575° — and it is still short by a factor of thirty.
Whatever bends sunlight by 17° at the Arctic Circle and by nothing at the zenith must squash the mid-afternoon Sun into an oval by about a quarter. It does not, so nothing is bending sunlight by 17°. That is the test, and it does not care what the bending mechanism is.
The reply usually made at this point is that it is not atmospheric refraction at all — “electromagnetic acceleration”, light curving over a flat plane by some property of light itself, or a dome whose optics bend sunlight like a lens: a mechanism no barometer can price, so the magnitudes above are said not to apply. Grant it. Grant the air, or the dome, or the light itself, whatever bending the flat model needs, and refuse to argue magnitudes. The escape still fails, in a way anyone with a camera can check, because bending that large cannot be uniform. It has to be 17° at the Arctic Circle’s 23.7° noon sun and essentially nothing at the zenith, or the tropics would be as badly wrong as the Arctic. That is a mean gradient of about 0.26° of bending per degree of elevation.
Now put the Sun in it. The disc is 0.53° across, so its lower limb sits in air that must bend light 0.14° more than the air its upper limb sits in. The Sun would be squashed vertically by 26 per cent — visibly oval — every mid-latitude afternoon, not only at sunset.
The real differential across the disc at that elevation is 0.17 per cent. The Sun is round.
No mechanism escapes this. The oval follows from the bend varying with elevation, and the bend must vary with elevation or the tropics would be as badly wrong as the Arctic; the name given to the cause — air, field, dome — never enters the calculation. A filtered photograph of the Sun at 24° elevation, measured for height against width, is the whole test. Celestial Globes Exposed runs the same test against the bent starlight that chapter needs.
Computed from Bennett at the two limb elevations, 23.44° and 23.97°: 2.277′ against 2.221′, a difference of 0.056′. Everyone has seen the effect this argument requires — it is what flattens the Sun into an oval at the horizon, where refraction really does change fast with elevation. The flat-plus-bent-light model needs that flattening, at a hundred and fifty times its observed strength, in the middle of the sky. Solar diameter taken as 0.53°; the annual variation of a few per cent does not move the comparison.
Q12 does not name an elevation. It says shadow angles, and this page answers the noon case, where the arcsecond figures hold. Serious shadow-angle work is done at local noon because that geometry is the well-behaved one, so restricting to it is not circular — but it is a choice of the favourable regime. p. 202 names refraction as the mechanism, which settles that branch, but it does not specify an elevation either. If the argument is specifically about long shadows near sunrise and sunset, the concessions get considerably larger and this page needs rewriting around Hirt’s k = −4 to +16 rather than around noon arcseconds.
His table is read off a page image, not extracted text. The p. 202 figure is a raster, so its six rows were transcribed by eye from the rendered page. They recompute exactly, but a misread digit is possible in a way it would not be with extracted text.
Horizontal homogeneity is assumed and unquantified. Every figure here assumes an atmosphere stratified horizontally. Real atmospheres have fronts. We could not find a measured bound on the residual from horizontal inhomogeneity at high sun elevations and do not assert it is zero. It is very unlikely to reach degrees; “unlikely” is not a number.
Several inputs are secondary-sourced. The 240 BC obliquity, the ancient station latitudes and Picard’s mean arc latitude are standard or approximate figures not traced to primary sources. Each is flagged at the point of use.
What would change our mind, specifically and in advance:
Two questions a surveyor would ask of section 3 — if k swings from −4 to +16, why does the profession still use one standard value, and how does refraction sit against the curvature term it is always paired with? Neither bears on the shadow argument; both are answered here because p. 202 borrows the constant.
Because it is the right value in the overwhelming majority of the work, for two documented reasons, neither of which is that the variation is exaggerated.
First, 0.13 was never a near-ground number. It is a dry-adiabatic value belonging to what the literature calls the intermediate atmosphere, from roughly 20–30 m up to a hundred metres, where temperature gradients settle near −0.01 K/m. Measured up there it is well behaved: Brocks got k between +0.10 and +0.12 from gradients on Nanga Parbat, and Mavridis and Papadimitriou got +0.12 to +0.20 shooting between two hills in Greece. Hirt’s own review of that work concludes there is “fairly small variability of k in regions well above the surface.” What his experiment shows is that the value does not survive being taken down into the first two metres, and he says so bluntly: the Gaussian 0.13 “is not suited for describing refraction effects in the lower atmosphere.”
Second, survey practice keeps the sight lines short enough that it cannot matter. The refraction term grows as the square of the distance — Δh = k D² / 2R — so a cap on sight length crushes it, and every specification opened here imposes one. The USGS trigonometric-levelling manual tabulates the refraction correction as literally negligible at 100 ft and 0.0009 ft at 500 ft, and instructs crews to keep levelling lines under 500 ft. WSDOT caps differential levelling sights at 200 ft. Kharaghani’s simulations put the boundary at 150 m of sight with a metre of ground clearance to stay inside Canadian first-order levelling specifications, 250 m by the reciprocal method.
| Sight length | Refraction term at k = 0.13 | If k were 16 | If k were −4 |
|---|---|---|---|
| 30 m | 0.009 mm | 1.1 mm | −0.3 mm |
| 60 m — WSDOT levelling cap | 0.037 mm | 4.5 mm | −1.1 mm |
| 150 m — USGS 500 ft cap | 0.24 mm | 29 mm | −7 mm |
| 250 m — Kharaghani reciprocal limit | 0.64 mm | 79 mm | −20 mm |
| 800 m — Hirt’s baseline | 6.5 mm | 804 mm | −201 mm |
Computed from Δh = k D²/2R; at k = 0.143 the same formula reproduces all six rows of the USGS manual’s own table to the last printed digit. Inside the sight lengths the specifications permit, even Hirt’s most violent excursions are worth millimetres. At his 800 m they are worth most of a metre.
The counter-case, from the other direction: Hennes finds that at only 100 m, with a −0.2 K/m gradient at sightline height, using 0.13 leaves a zenith-angle error of 0.63 mgon (a milligon; 400 gon make a circle) — which exceeds a modern instrument’s accuracy, and which she says is commonly met in summer over heated ground at the usual 1.5 m sightline height. So the default is comfortable against a millimetre height budget and uncomfortable against an angular one. Which tolerance you are working to decides it.
Refraction never travels alone in survey work. The Earth’s curvature drops a distant target by D² / 2R and refraction lifts it back by k D² / 2R. Both go as the square of the distance, which is the whole reason instruments carry one “curvature and refraction” toggle rather than two:
Their ratio is k, and it does not depend on distance at all.
So in the standard case refraction is a thirteen per cent trim on curvature — the “about one seventh” the USGS manual quotes — and the two collapse into a single combined correction of (1 − k) D² / 2R. What changes with distance is not the balance between them but whether either is worth measuring.
| Sight length | Curvature drop | Refraction lift at k = 0.13 | Net | Verdict |
|---|---|---|---|---|
| 30 m | 0.07 mm | 0.01 mm | 0.06 mm | neither matters |
| 100 m | 0.79 mm | 0.10 mm | 0.68 mm | neither matters |
| 250 m | 4.9 mm | 0.6 mm | 4.3 mm | curvature does |
| 1 km | 78 mm | 10 mm | 68 mm | curvature dominates |
| 10 km | 7.85 m | 1.02 m | 6.83 m | curvature dominates |
Same formula as above with k = 1 for the curvature column. At k = 0.143 it returns 0.667 ft of curvature and 0.095 ft of refraction at one mile, against the USGS manual’s printed 0.667 and 0.0953.