Gauss: The First Data Scientist?
Posted on
We think of data science as a thoroughly modern discipline. Indeed, it may conjure images of powerful computers, vast datasets and complex models and code that make predictions at scale. But at its core we take noisy, often incomplete observations, and fit a mathematical model to them so we can then use that model to make future predictions.
This idea isn’t new. In fact, one of the most dramatic demonstrations of data science in action took place more than two centuries ago, when a 24-year-old German mathematician used a handful of observations to find a lost planet.
In this blog we will discover how Carl Friedrich Gauss tracked the motion of the dwarf planet Ceres, pioneering the use of one of the most important tools in modern statistics, and laying the foundations for modern data science.
Who was Gauss?
Gauss has the moniker the "Prince of Mathematicians", and it is difficult to overstate his influence. Over his lifetime, he made fundamental contributions to number theory, geometry, astronomy, magnetism and statistics. His name is attached to ideas that students still meet today: Gaussian elimination for solving systems of equations, the Gaussian (or normal) distribution that describes the famous bell curve, and Gauss's law in physics.
Gauss was a child prodigy. One oft quoted (but probably inaccurate) story is that his schoolteacher set the class the task of adding up all the numbers from 1 to 100, expecting to keep them busy for a while. The young Gauss spotted that the numbers could be paired (1 with 100, 2 with 99, and so on) with each pair summing to 101, and produced the answer of 5,050 almost immediately.
However, it was an astronomical puzzle that made Gauss famous across Europe, and the way that he solved it could be seen as one of the earliest applications of what we now call data science.
The disappearing planet
On 1 January 1801 the Italian astronomer Giuseppe Piazzi spotted a faint object moving against the background stars. Astronomers had long suspected that a "missing planet" might lie in the suspiciously large gap between the orbits of Mars and Jupiter. Piazzi's discovery, which he named Ceres after the Roman goddess, seemed to be exactly that. However, Piazzi only managed to track Ceres for 41 days, recording around two dozen observations, which covered a small arc of its orbit. He then fell ill, and by the time other astronomers tried to follow up, Ceres had moved close to the Sun and vanished from view.
To find Ceres again, when it emerged on the other side of the Sun, astronomers needed to know its full orbit. But calculating an orbit from such a short, imperfect set of observations seemed impossible. The measurements were sparse, covering only a small proportion of the orbit, and all measurements contain errors. The leading astronomers of the day made predictions of where Ceres would reappear, searched the skies, and found nothing.
Enter Gauss
Gauss, then 24 years old and primarily known as a mathematician, took up the challenge. He established the standard approach to this type of problem by combining two distinct mathematical techniques. As we know from our previous blog on applied maths, the first thing he needed was a mathematical model. From the earlier work of Kepler and Newton, the mathematics of how planets move around the Sun was well understood, and an orbit can be completely described by a small set of parameters.
The second was a principled way of dealing with error and fitting these parameters. To do this Gauss asked, “which orbit makes the differences between the model's predictions and the actual observations as small as possible overall?”. His answer was to choose the orbit that minimises the sum of the squared errors, the method we now call least squares.
After a period of intense calculation, Gauss published his prediction of where Ceres should reappear. It differed substantially from everyone else's. On 7 December 1801, the astronomer Franz Xaver von Zach pointed his telescope at the position Gauss had calculated? and there was Ceres, very close to where Gauss had predicted. The recovery of Ceres’ position in the night sky was confirmed on New Year's Eve, almost a year to the day after Piazzi's original discovery and Gauss became internationally famous.
What is least squares?
The beauty of the least squares method lies in its simplicity. Consider a problem similar to Gauss’s: you have a set of data points and a model with free parameters? for example, a straight line through a scatter of points, where the parameters are the line's gradient (slope) and intercept. For any choice of parameters, you can measure how far each data point sits from the model's prediction. We call these differences residuals.
Under least squares we square each residual, add them all up, and choose the parameters that make this total as small as possible. Squaring ensures that errors above and below the prediction don't cancel each other out, and it penalises large errors much more heavily than small ones.
Gauss also showed that (assuming a normal distribution of errors) squaring them mathematically guarantees that the resulting model is the most statistically probable representation of the data. In our previous blog on machine learning, we described how models “learn” by minimising a loss function that measures the difference between predictions and data. Least squares is precisely a loss function. In fact, the squared-error loss is still one of the most widely used in machine learning today.
Final thoughts
Was Gauss really the first data scientist? Purists might object and argue that data science as a discipline only emerged in recent decades, and that Gauss would have called himself a mathematician and astronomer. Others have made compelling claims on behalf of later figures, for example John Tukey, who championed data analysis in the twentieth century. However, if data science is about extracting knowledge and providing robust predictions from imperfect data by combining mathematical models, statistical reasoning and computation, then the recovery of Ceres in 1801 is as good a founding moment as any.
More than two hundred years later, every time a regression model is fitted, a loss function minimised, or we train a new machine learning approach on data, we are standing on Gauss's shoulders.
Related Blogs
Disclaimer
The opinions expressed by our bloggers and those providing comments are personal, and may not necessarily reflect the opinions of 好色先生TV. Responsibility for the accuracy of any of the information contained within blog posts belongs to the blogger.
Back to blog listing