Some thoughts on organizational complexity
Thinking about complexity #
Knowledge Representation is a central topic in the field of artificial intelligence. Research in this area tackles the challenge of mapping the external world into simplified, machine-readable models. Creating a sufficiently sophisticated model is as important, if not more, than the algorithmic processing that occurs once the model is built.
The more external states that can be represented in a model, the more complex we could say it is. Simple models can cause people to falsely believe that successful results can easily be “scaled up” to more complex models. Unfortunately, it’s rarely that simple.
A well-known example of this in AI was the early promise demonstrated by the SHRDLU project:
The result was a tremendously successful demonstration of AI. This led other AI researchers to excessive optimism which was soon lost when later systems attempted to deal with more realistic situations with real-world ambiguity and complexity.
To summarize: don’t assume that a successful AI project which utilizes a simple model will seamlessly scale up to more complex circumstances.
The complexity of large companies #
It’s the job of a large company to understand what the outside world wants to buy, and then mobilize their corporate infrastructure into delivering the right product, at the right place, at the right time.
For the sake of illustration, let’s use Apple as a stand-in for a large company. A single human being could never understand all of the constituent pieces of work that goes into shipping the hardware and software components contained in a new iPhone. But no single Apple employee needs to understand all of the pieces. Apple has a number of well-run subsystems which each tackle a piece of the process that ultimately manifests as a white box with an iPhone 5 in it. Apple is very complex organization… but that complexity is what allows them to ship very complex products.
There is, of course, a downside to organizational complexity. Complexity tends to breed more complexity. When you talk to employees of a large company, it seems as if their entire world is made up of byzantine internal politics with no relation to customers in the external world… a large percentage of every day is spent attempting to navigate ever-shifting politics surrounding the org chart. It would seem that the larger a company gets, the more the company tends to allocate resources towards inwardly focused (as opposed to outwardly focused) issues.
Organizational complexity can show up in odd ways in a shipping product. For instance, why does a single product, the Microsoft Surface, give consumers the option between two very different OSes: one model with “Windows RT” and another with “Windows 8 Pro”? I am sure there is an official reason why this is Totally A Good Idea, but it mostly seems like Microsoft decided to ship their org chart.
Thinking about organizational complexity from the perspective of an insider, the “Peanut Butter Manfiesto”, written by a Yahoo SVP in 2006 remains an excellent first-hand account of what he witnessed at Yahoo.
Complexity in startups #
Early-stage startups aren’t very much like large companies, and that difference certainly applies to organizational complexity. Nonetheless, a startup with one or two employees must create some sort of internal model of the external world regarding what do people want, when is this ready to ship, what is the best technology for us to use to deliver our product, and so on. Even with essentially zero bureaucracy, it’s easy to get hung up on inwardly-facing projects that don’t actually matter in the long run.
When we talk about things like “focus” and “simplicity” and “doing just one thing right”, we are more or less discussing complexity management. If a company tries to do too many things at once, the company org chart and workflow will naturally begin to model that complexity.
From the perspective of a startup, having tools at our disposal such as AWS, outsourced HR, or streamlined billing providers let us keep our internal model of what it is we are supposed to be doing with our time as simple as possible. Simple models are the easiest to understand and modify on the fly.
When complexity in a startup is good #
As a startup grows, it becomes necessary for the scope of the internal model to grow so that it can effectively serve the demands of the outside world.
The simplest example of this is a dramatic increase in traffic. If a startup gets 100x more traffic, then the organization is faced with an immediate, overwhelming amount of work required to keep their service operating. So the startup spends time creating a more sophisticated hosting setup, or implements database sharding, or hires someone with an expertise in scaling. What once was as simple as rolling out some code to Heroku is now something with far more moving parts. In this example, the more quickly and effectively the startup recognizes and adapts to the newly-introduced external complexity, the better.
To take another example, a startup generating increasing revenue/sales must intentionally create additional internal complexity. To deal with increasing revenue, an organization must create/refine the internal process regarding things like customer support, sales support, refunds, taxes, billing and accounting. Even if the decision is to outsource all of those functions and take on zero additional headcount, there is still systemic responsibility for those functions to be adequately performed and managed. A founder probably shouldn’t say: “I believe in simplicity and focus, therefore we will have no systems in place to handle accounting or customer support.”
A successful increase in product market fit, combined with a successful increase in organizational complexity to handle increased demand, can feed on itself to create a virtuous cycle of growth. This virtuous cycle is how, once in a blue moon, a startup can metamorphize into a large company.
Digesting complexity #
Whenever I deal with what feels like an overwhelming problem, or get spun out trying to make sense of the myriad potential things I could be doing better, or hear about a company having a tough time because of things like a founder falling out, lack of focus, employee infighting, bad hires, can’t hire fast enough, hired too fast, etc, I can’t help but think of this quote:
More companies die of indigestion than from starvation
Years ago, when I first heard this quote, I dismissed it as a useless platitude. However, the older I get, the more I have come to appreciate the wisdom contained within that short and simple statement.