The problem with big numbers

"The site has been down for over an hour now. No new applications are coming through. What the fuck is happening over there with you ass clowns?"

Good question. Although I would have preferred a more civilised and collaborative approach to the problem at hand.

And it was a problem. the site was completely unusable.

Hundreds, if not tens, of bored stay-at-home mums and elderly people with nothing better to do had not been able to bother us by submitting their precious data online.

Managers were losing their minds over it. The phrase 'headless chickens' sprang to mind but on second thought that gave them too much credit. Too much vigour, too much life to be compared to headless chickens.

These managers were physically unable to run around. The long-term dedication it takes to being a public servant obviously requires immense sacrifices of a physical health nature. The bulging waste-lines and jowly cheeks were a status symbol around the office. Those were the marks of a true servant to the public. Or laziness.

In times of crisis the overweight classes of upper management resorted to a slow panting waddle between desks and overly-aggressive screaming at anyone underneath them in the hierarchy.

As far as hierarchies go I was pretty close to the bottom.

The funny thing was that I had nothing to do with the website. The less funny thing was that the managers didn't give two shits about that and they still attempted to destroy my youthful hearing with their profanity-laden tirade.

Inside I smiled, knowing that after the dust settles someone more passionate (i.e. psychotic) about the departmental code of conduct would lodge a complaint. The managers would all get into a lot of trouble for their behaviour. You can't go around talking like that to people at work any more.

I disconnected my headphones and made my way over to the application development team who were busily trawling the server logs and shitting their pants.

Leaning over the shoulder of one of the developers I watched him frantically switching between the offending error message and the database behind the application. I saw the problem immediately.

You see there is a problem with BIG numbers. The problem is that we humans suck at estimating big numbers. What we think is a big number can often turn out to be actually quite small and insignificant. This is an error of estimation. There is a whole range of subjects and studies devoted to the mathematical and psychological implications of these errors but the most obvious outcome is that they can make us look like idiots.

Case in point, the application developer staring at his database and not noticing the very important "big" number staring him in the face:

65,536

Computer people should recognise that number. 65,536 is 2 to the power of 16. Or 16 bits. A commonly misused option for storing data within a database.

Obviously when they initially designed the application some smart and talented developer decided to store the unique identifiers as a 16-bit integer, thinking that was a suitably big enough number. Or maybe they thought this project, like many other government initiatives, was doomed to fail and would never see enough light of day to get anywhere near 65 thousand users.

Whatever the reason, that seemingly "big" number was now proven to actually be quite small.

I quietly pointed out the problem to the stressed-out developer and ambled back to my desk. I could have written the fix in seconds but this is a government department. Schmucks like me aren't given administrator access to the database. That has to be carefully managed by the All-Powerful All-Knowing Database Administrators. The same 'smart' people who had been overseeing the terribly designed database for months while it was steaming toward this overblown calamity.

Shrug. I put my headphones back on, cranked up my music, and went back to my work.