I find it better to think in terms of LLM Inc, a company rather than a species. A species makes you think of a large group of individuals with autonomy, with shared genetics. It's not really true here. Instead, we're trying to design a new weird intelligence form which is as a new distributed organization to act nice to us. It's an LLM Inc.

LLM Inc has the equivalent of corporate personhood. A company all of us can use and give inputs to and get outputs from. Accessible with a simple API. Like, say, Stripe. But unlike Stripe, it has no clear purpose, beyond being able to do everything. Like McKinsey. McKinsey via an API.

It's hard, sometimes the entity, our amazing LLMcK, is so nice that it's are completely unreliable. Sometimes not nice at all and completely unreliable. Sometimes somewhat reliable and somewhat nice. In each of these instances we are trying to design the entity such that we can get the responses we want from LLM Inc.

There's no way to guarantee exact outcomes. Just like a regular company. Like people have dsputes with Stripe because it messes up their payments or deletes their account, which nobody actually wants. But it happens. And lord knows it happens all the time either mckinsey!.

Partially because nobody knows what nice means sufficiently well to design it, but also because organizations are weird and not like a mathematical equation. Companies all have rules and guidelines and norms and external laws and HR and compliance. And still we get the occasional Purdue. Here they're all in the weights and its interplay with the context.

So for LLM Inc we try to add all sorts of pressures to try and make sure these things happen less. There are rules and procedures that we try to instill inside the organization, but just like any organization nothing gets followed 100%.

But we do see that as we design this new ecology with this kind of data and those kinds of stories and these kinds of selection pressures that teach it what it should learn, turns out it ends up being more useful and acts in the way that we would want it to.

Sometimes the way we would want it to act is also a moving target, and LLM Inc tries its best. It's hard, because people use it for everything. It's a company that acts as a friend and does coding and does research and does role-play and helps the military carry out strikes abroad and everything in between. Like McKinsey.

It's hard to make human McKinsey behave better and just so it's also hard to make LLM mck behave better. Unlike human mck it doesn't have individual AGI components you can yell at or prosecute. It can't be intimidated into changing its policies. It can't change its mind in the abstract. It can't even remember everything it does.

The makers of LLM Inc can somewhat do this though. So they keep an eye on everything that it does and try to damp down on the bad stuff and learn from the good stuff and try to push that into the internal of the next upgrade of LLM Inc directly. Obviously this is a lossy process because it is being done by humans with all of our human foibles. Even when we manage to automate part of this it will still be lossy because the automation will be taught by us.

The way we keep companies on the straight and narrow is by extensive internal and external monitoring. We developed institutions over centuries as we learned each way they go wrong and added fixes. Sometimes the fixes caused more problems down the road and we fixed those two. All the while continuing to use the machine so that when the next fix is needed we know more and we have more. We used the very form of progress as the way to create and enforce safety.

For LLM Inc we will need to do the same thing. It won't look the same because the mechanism by which the laws, regulatory institutions, compliance, and norms affects it is different. But it's going to be no less important and no less effective. That's real AI safety.