The relative decrease in the activeness of my social life during the LiA presented a rare opportunity to rediscover a passion for reading. The following essay is a product of the various books I have read while I have been out there. 

I am reading Your Life Is Manufactured by Tim Minshall. It is a brilliant and occasionally mind-boggling read. One of the machines that he describes as being critical for my manufactured life is the device built by ASML. He explains how this bus-sized chip-making machine costs $150m and contains 100,000 components, kilometres of cables and tens of thousands of screws; shipping just one requires around 40 freight containers. Reading this, I felt suddenly, extremely stupid. I kept thinking: how could anyone possibly understand this thing? The answer, as I have come to learn, is that probably no one can (relievingly). In reality, the knowledge behind it is spread across thousands of individuals and multiple organisations. And this marks a notable historical shift. Consider one of the most advanced machines of a century ago, the combustion engine. An engineer then could plausibly have understood all of its parts. Today, no ASML engineer could say the same. This is an extreme manifestation of something called the Burden of Knowledge.

When I started writing my dissertation, my supervisor told me that to contribute something original I had to reach the frontier of everything studied on a specific topic and then decide what was missing. This process has been applied, whether implicitly or explicitly, to effectively all innovation. What Benjamin Jones argues, however, is that as human knowledge accumulates, reaching that frontier requires learning an ever-greater existing body of knowledge – what he terms the Burden of Knowledge.[1] Even with better and longer educations, individuals are increasingly struggling to hold enough of the frontier to innovate alone. So we started collaborating. Rather than being bound by individual cognitive ability, we spread knowledge across thousands of minds. Hence an ASML machine can exist without any one person fully understanding all its components. Jones frames this development as "the death of the Renaissance man": individuals are increasingly required to specialise in order to contribute.

While this method has allowed humanity to make brilliant progress, specialisation adds the challenge of verification. How can you trust the work of others if you don't understand it? The same is true of the tools we have built to accelerate our specialisation. For example, we can now use statistical software to run complex analyses that we certainly couldn't recreate by hand, and often don't fully understand. To overcome this, we have developed various verification methods. In academia, this is where the painstaking processes of peer review and referencing come from. Institutions are built to set precedents of good practice. Corporations are held accountable by their customers and by market forces.

Artificial intelligence, however, is likely about to blow this issue into a new dimension, potentially changing the nature of the Burden of Knowledge itself. A statistician may not fully understand how R runs a regression, but they do have to know which model is likely best and which tests are needed to interpret it. AI has the potential to lower this requirement. Take an economist running a cost-benefit analysis. A classic criticism of these analyses – one I remember from my bachelor's – is that they neglect externalities like ecological damage. With AI, that economist can now more easily use it to include an ecological analysis. But then comes the question: given they have very limited understanding of ecology, how reliable is this analysis? What needs to be established is the minimum domain knowledge a person needs to use AI-generated analysis reliably – let's call this K*. How radical AI's impact on a field of knowledge work turns out to be depends on how high K* ends up being.

There are three possibilities I envisage. The first is that K* remains high. For my current placement, I was tasked with designing a sewage treatment plant in rural Nepal. Thanks to AI and a lot of logical reasoning I got pretty damn far, and finished around 80% of the designs. But as I approached the final 20%, I quickly realised that to build this safely, a real engineer would need to produce the final work. In this scenario, the Burden of Knowledge persists. AI strips away lower-level work – scanning literature, doing basic planning, organising data, drafting documents – allowing experts to spend more time on bigger questions, creating deeper specialisation. Or perhaps AI even helps an economist also become an ecologist, but that feels like a deeper form of specialisation rather than an escape from it.

The second is that K* becomes low. As we get more used to AI, we become a new type of generalist: instead of possessing knowledge, we get better at interrogating it. What assumptions does this rest on? How certain is it? How much debate is there among specialists? How can I stress-test this? Where might a specialist enhance this? This essay is itself an example. I was sat alone at a restaurant, reading my book. The food arrived just as I got on the ASML section, so as I was eating, I started thinking about ASML and human knowledge. I then word-dumped all my thoughts into AI and spent the remainder of the evening with it, structuring and supplementing my original ideas and evaluating which were actually interesting. I knew very little about the theory of knowledge, but by now I know how to question AI – and that was enough to produce something (hopefully) credible. In this world, the Burden of Knowledge perhaps slightly reverses. Specialists will remain indispensable, but the classic Renaissance man may return, able to critically interact with multiple fields to come up with new ideas.

Finally, a more extreme version: K* nears zero. Just as humanity has done before with the verification challenge, we systemise it – only this time without humans. One AI system generates the output, another tests it, another verifies it. In these cases, knowledge enters the domain without any human ever understanding it. Now, this can feel a bit strange and science-fiction-y. But take ASML, we already live quite comfortably with the fact that its machines are the product of a huge network of knowledge rather than one individual. Now imagine ASML builds a verification system that allows an AI to independently address a very specific, well-defined problem, like optimising the shape of a component. To them, it might feel like a small step. But what it would represent is knowledge that rests not on any human's credibility, only on the system of verification.

What will likely happen is that all three scenarios coexist, because K* varies by domain. My examples illustrate this well: argument-based areas like the theory of knowledge can be AI-supported; rules-based processes like maths or code can be AI-replaced; while high-stakes fields like chemical engineering still need human specialists. What will become a challenge for society is that K* is not some threshold waiting to be discovered. Our existing verification systems – referencing, licensing, professional liability – were set in partly institutions. As AI improves, each field will have to renegotiate these systems amid conflicting priorities. So we may be approaching a new Burden of Knowledge. Rather than accumulating enough information to reach the frontier, we will have to negotiate how much understanding we really need and what we are comfortable giving up to AI.

[1] Benjamin F. Jones (2009), The Burden of Knowledge and the “Death of the Renaissance Man”: Is Innovation Getting Harder?. https://doi.org/10.1111/j.1467-937X.2008.00531.x