## Benford’s law is number one

2 Comments »Posted on Wednesday 3 December 2008 at 10:56 pm by Jacob Aron
In Mathematics

Football statistics. The lengths of roads in Britain. Fundamental physical constants. What do all these groups of numbers have in common? The answer may surprise you, but it is this: in all three data sets, numbers that begin with 1 are far more common than those whose first digits are 2, 3 and so on up to 9.

Now, you might expect all numbers to start with 1 to 9 equally. As there are nine numbers you would think that chances of any number beginning with 1 would be 1/9, or around 11%, but actually it is more like 30%! So what makes these groups so special?

It’s a bit of a trick question actually. It turns out that for many large sets of data, numbers that start with a 1 crop up around 30% of the time. Numbers that start with a 2 occur around 18% of the time, and the probability decreases with each successive number. It’s not just in the examples I listed above that this happens, but also stock exchange data, population figures, and many more. This seemingly strange phenomenon is all thanks to Benford’s law.

It was first observed by a man named Simon Newcomb in 1881. As both a mathematician and an astronomer, he often used logarithms in his work. The logarithm of a number in a particular base is the power to which that base must be raised in order to produce that number. It’s easy to illustrate with an example: in base 10, the logarithm of 1000 is 3, because 103 = 1000. We write that log10(1000) = 3.

Logarithms can be used to make complex calculations much simpler, as long as you know how to convert back and forth to regular numbers. Nowadays we can let a computer do all the work, but back in Newcomb’s time people were forced to rely on weighty tombs of pre-calculated logarithms. One day, Newcomb realised that the pages of the book he was using became more worn the closer you were to the front. He came up with a formula that described the probabilities, as shown by this handy graph:

Benford's law in action: lower leading digits are far more common

Why then do we call this Benford’s law, if Newcomb came up with the formula? Well, Newcomb dismissed the idea an ideal curiosity. It was forgotten until 1938, when physicist Frank Benford noticed the exact same occurrence. He decided to investigate, and gathered masses of data to see if the rule was universal. Looking at sets similar to the ones described at the start, he found that it was really true: when it comes to large amounts of data, not all numbers are created equal.

Of course, you have to be careful when applying Benford’s law. A list of secondary school pupils and their ages will not follow the law, since all pupils must be aged 11-18 the probability of a leading 1 is 100%! On the other end of a scale, a collection of dice rolls will show that each number has a 1 in 6 chance of appearing; dies are truly random. Benford’s law will only apply in cases that fall somewhere between these two extremes, but thankfully this still includes a lot of data.

An interesting fact about Benford’s law is that it applies no matter the units of measurement used. You can measure your roads in miles or kilometres, and Benford’s law will still apply. This is known as scale invariance, and can actually be used to mathematically derive Benford’s law. You can work out which distributions of first digit probability stay the same when you switch from miles to kilometres, and it turns out there is only one: the formula that Newcomb came up with.

Benford’s law is a fascinating mathematical fact, but surprisingly it also has practical applications. Get this: it solves crime. No, really. If you’re a crooked accountant who likes to cook the books, Benford’s law will catch you out if you aren’t careful. If our dodgy dealer doesn’t know about Benford’s, they will probably pick numbers at random, or tend to stay in the “middle” (4, 5, 6, 7). Either approach will result in a data set that looks perfectly reasonable at a casual glance, but an analysis with Benford’s law will reveal that not enough numbers start with 1.

There you have it. Benford’s law: a mathematical oddity that just happens to have its uses. For fun, I thought I’d see how Just A Theory page views stack up against Benford’s law. WordPress can track how many times each page has been visited, so I grabbed the first digit from each of these numbers. The results:

The Benford's law prediction (blue) closely matches the page views (red)

It’s not perfect, but Just A Theory doesn’t have that many pages (yet!). As with anything statistical, a larger data set will get you closer to a mathematically predicted distribution. Still, it’s pretty impressive to see Benford’s law in action. Maybe I’ll try again in a year’s time, when I have more data!