This is an excerpt from the draft of my upcoming book on great founder theory. Learn more here.

Why was Barack Obama elected president in 2008? Was it because he ran a smart and successful campaign? Was it because new social media sites allowed young people to get interested in politics? Or was it because American culture was generally shifting away from George W. Bush’s brand of conservatism?

If you read the news articles published on November 4th, 2008, you’ll notice something interesting: journalists explain this historic event in many different ways. Some journalists attribute the campaign’s success primarily to the individual leading the campaign. Others focus more on the influence of new technologies on campaigning. Still others explain it in terms of a general cultural or political shift.

These explanations are revealing—not necessarily of what actually landed President Obama in office, but rather of how each individual journalist conceives of the way things happen in the world. Through their explanations for the outcome of the election, we can glean a bit of their implicit theories of history.

Concept & importance

A theory of history is an explanation of how things generally happen in the world, both in the past and in the future. If, for example, you subscribe to the great man theory of history, then you might explain events by looking at the influential individuals who shaped them. If you subscribe to a technological determinist theory, on the other hand, you might explain events in terms of the technologies that allowed them to occur. Or, you might subscribe to a social determinist theory, explaining both influential individuals and new technologies as the makings of greater societal forces.

Someone operating under the great man paradigm might explain Obama’s election as a product of his and his staff’s exacting efforts in the day-to-day of campaign work. A believer in technological determinism might attribute the win to the unprecedented use of social media, which mobilized previously uninterested voters. Someone adhering to a social determinist view might draw a straight line from the Civil Rights era to the election of Barack Obama, pointing out the inexorable cultural shift towards empowering African-Americans.

Everyone has a theory of history, in that everyone has an explanation of why the world is how it is and an understanding of how the world changes and has changed. Everyone has to: without an understanding of how the world works, no matter how faulty, implicit, or subconscious, we would be prohibited from acting in what we believe is the right way to achieve our goals, whether big or small. Few people could tell you plainly that they are social or technological determinists, or adherents of great man theory. But everyone, if asked, can give reasons why some event or another happened, and whether, or why, it might happen again in the future.

We don’t just explain things with our theories of history. We act on them. If you believe that individuals have the power to significantly shape history, for example, you might be more inclined to make things happen yourself. If you believe that technology drives historical change, you might specifically try to invent new technologies. If, on the other hand, you believe that the fate of the world has already been decided, or if you believe that history is inevitably heading in a certain direction, you may be less inclined to take a stand. After all, if it’s going to happen, then it’s going to happen.

Therefore, whether we’re trying to change the world in a major way or just live our lives in society in the best way possible, it’s vital that we come to understand the true theory of history. We need the true theory of history in order to take the right actions in the world, and we need to accurately predict the results of our actions. If we have an incorrect theory of history, we run the risk of producing unknown and possibly catastrophic consequences, for ourselves or others.

It’s important here to note the distinction between the true theory of history, and the “true” theory of history that we’re aiming for. The true theory of history will be unmanageably complex, because the number of factors that actually influence what happens in the world is incalculably  large. Because of its complexity, the true theory of history will be difficult, if not impossible, to use to explain what’s going on in the world. In aiming for the “true” theory of history, we are assuming the power law: we are assuming that there will be a small number of factors that have disproportionately large effects on the world, or that can explain the existence of other factors. We are aiming for a theory that generally explains how things happen in the world. Going forward, we will drop the quotation marks and stipulate that the true theory of history is the theory that takes into account the core causes contributing to the world as it exists, making it comprehensible and usable to us mere mortals.

No one has it

No one in the world has figured out the true theory of history. If they did, we’d know: not only would they be extremely, visibly, powerful, but they would be active in many domains—politics, religion, culture, technology—reshaping society step by step, or taking seemingly prescient advantage of trends, with many successes and few false starts. There are historical examples of incredible individuals, such as the Indian Emperor Ashoka the Great, and organizations, such as the Catholic Church, whose repeated success across multiple domains is difficult to explain without them understanding at least fragments of a true theory of history.

There are many reasons why no one has figured out the full true theory of history, some psychological and some practical.

There are at least three psychological reasons for why most people are deterred from finding the true theory of history. The first is that the vast majority of people only have an implicit, or subconscious, theory of history. In other words, most people do not even have the concept of a theory of history. The problem with relying on your implicit theory of history is that it’s wrong, without a doubt. The world is complex, and your theory of history has to explain how everything in the world works. Without explicitly trying to improve your theory of history, there is no hope: there will be countless things that you have not had the time or perspective to take into account. Improving your theory of history implicitly is not systematic enough to work.

The second reason why no one has managed to achieve the true theory of history is that many people endorse one theory of history while unknowingly acting on another. For example, some people explicitly endorse the technological determinist view of history even as they implicitly act on the great man paradigm: believing that it will require the work of remarkable individuals to create the technology that will save the world, for example, instead of believing that the inevitable, impersonal progress of technology will do so.

There can be many belief-based reasons for why people fall into this trap, but on a more basic level, people simply don’t have a good sense of what their implicit theories of history are, or know how to explicate them, which means they cannot reliably align their intellectual and emotional beliefs. To some extent, acting on your implicit theory of history while operating under a different explicit theory is fine — after all, your implicit theory will, for a while, be more nuanced than your explicit one. What is problematic is to act unconsciously on one theory of history and proclaim another; this makes it very difficult to improve your implicit theory of history, which you act on.

The third reason is that people tend to switch between theories of history in an unprincipled way, which prevents them from noticing theory-threatening anomalies. If they can’t notice and explain seeming anomalies in their theory of history, then they can’t improve their theory. If someone largely adheres to the great man paradigm, for example, but resolves any contradictions by falling back on the technological determinist view, then they’ve prevented themselves from justifying their understanding of the great man theory, or realizing that their justification is inadequate or incorrect. Theory-threatening anomalies have to be resolved, not rationalized.

These are just a few of the psychological barriers that prevent people from making progress towards the true theory of history. But there’s a simpler, more practical problem: the world is complex. In order to understand it, you need the right methodology, and you need a huge amount of properly processed data.

More essays