Man vs Numbers: My five-week fight with 2,352 data points

Like Comment

On Monday, July 1st, 2021, I was an excited man. At the outset of my project, I thought I knew exactly what I wanted to research, exactly how to research it and exactly what my results would look like. I had six weeks to gather my data, run my tests, and proudly walk away from my project with metaphorical Michael Bay explosions behind me and a smile on my face.

Within a week, that belief had faded. I had carefully herded 780 data points into Microsoft Excel and eagerly expected my regressions to jump out of the screen, high-five me and confirm that I was on the right path.

As it transpired, I had ridiculously high standard errors and P-values; to non-statisticians, this is the world’s way of telling you that you are almost certainly wrong.

Demotivated, I rebuilt my project from the ground up. My search widened from a single country to the entire world, and I spent the next five weeks developing a horrifying 2,352-point spreadsheet incorporating 42 countries. I grappled with election systems (who chose to make Poland so complicated?!), trawled through government websites for COVID data, and fired passive-aggressive emails at uncooperative data firms.

Finally, results started taking shape, regressions began delivering, and I stopped seeing Rstudio error messages when I closed my eyes. By the end of the six weeks, I had something worth sharing.

It did, however, come at a cost. For weeks beforehand, I thought I was a terrible scholar; whilst the Laidlaw Scholars Network showed me geniuses of scientific discovery and leaders revolutionising communities, I was restructuring my entire project. Not until I spoke to older Scholars did I realise that I was not alone in this – even some of the most interesting and successful projects faced sharp turns and dramatic changes along the way.  

My message to the next generation of Scholars is simple: if your project needs to change shape, let it. Do not expect your work to be a clean run with game-changing outputs and cheering crowds at the finish line. Take the bumps and challenges as they come, and – above all – be confident in your work.  


Clarke, St Andrews

I am an undergraduate Economics and Mathematics student in the 2021 Cohort of the University of St Andrews. My research project will be a statistical analysis of the impact of the COVID-19 Pandemic on international voting behaviours, particularly regarding the re-election of political leaders (or lack thereof), but my interests expand to anything Econ.