Make money doing the work you believe in

ok since no one asked and I'm done with my breakfast I'll give you my lowdown of where we are - the very short tl;dr is that excitement about the board remaining steadfast and the new CEO is warranted BUT there will be a lot of fallout and the direction this goes into and ultimately whether this is a net positive or negative for p(doom) remains to be seen

The non-tl;dr version goes like this. Getting a CEO that is on-board with x-risk is great and the board not caving in to investor pressure is great and sent a strong message. However the board remaining steadfast is not an absolute win situation necessarily and certainly not a free lunch. In order to continue operating any company needs a good combination of the following: funding, product, staff, leadership, public trust/interest. Following Altman's departure OpenAI is going to take some serious hits on most of these:

  • Despite Microsoft's statements that they remain committed to OpenAI this isn't really worth the screen pixels it's written on. Funding will dwindle as Altman's new team gets up and running and produces a viable GPT competitor. It's not going to be immediate but imo it will inevitably happen. It's become clear to Microsoft that they can't acc or monetize OpenAI the way they want to. They can do whatever the hell they want with an internal product. The other aspect is of course ease of access to compute not just funding itself, which will also be a loss here.

  • In terms of product they remain the leaders for now. People aren't just going to stop using ChatGPT because someone else is CEO and Altman isn't going to pop out a leading model in a month. One hopes that they will be able to survive on that and keep innovating. Ultimately having them become research/safety focused is great but if they are not to become another MIRI (in the background with little funding) they need to remain commercially viable as well and appealing to investors, even if it's not big bucks daddy Microsoft or whatever.

  • There will clearly be an exodus of talent in terms of engineering and other areas. Intuition says that most safety conscious people will stick around, but even for some of those their loyalty/trust in Altman will supersede that and they will follow him to Microsoft. There are a lot of talented engineers out there but it's fair to assume that OpenAI had already skimmed off the best ones. New talent will take time to find and train. It will also require funding to be able to remain market competitive which taken against the previous point may become gradually more difficult

  • There's also going to be a drain of leadership quite obviously as well, the extent of that is similarly unclear yet but it will happen. I am somewhat more optimistic about the replace-ability of leaderships talent but that still hinges on funding and of course, we have to recognize that Altman himself was a very inspiring figure that drew in both talent and investment. It remains to be seen if Emmett will be able to match that and to what extent.

  • In terms of public trust things are debatable. Most people in the wider public won't even have registered that anything happened. Governments will have noticed for sure and on the one hand this could be taken as proof that governance worked, on the other it's impossible not to see chaos. One hopes politicians would take the best from both, but they may just ignore it as same old same old. In terms of the community that pays attention to AI it's pretty clear these will cause and have already caused some new and pretty significant rifts to occur. People are blocking each other on Twitter over it, various bridges where communication existed before are being burned, some maybe permanently. It's hard to say what impact this will ultimately have. But it will factor in.

Overall the best way I see this going is OpenAI manages to keep GPT going even if at a slower pace and retain a share of the market to still attract investment and maintain some level of funding even if considerably lower once Microsoft money go away. They focus primarily on safety research with small increments to existing products to keep business going. Microsoft stays invested for a little while to hedge bets but ultimately once Altman is up and running divests resources towards their own project. Even in this timeline is hard to see how much negative Altman getting free reign to push the pedal to the metal within Microsoft will be on p(doom). On the one hand Microsoft can’t afford to risk their business on a bad AI product that goes horribly wrong and they want safety; in principle Altman does too, in practice “it’s complicated”. On the other Microsoft wants money and speed and they have been reckless with AI product releases in the past. With Microsoft money and compute at their back the Altman team could go faster than they did at OpenAI and ruin all the positive that may come if OpenAI goes safety research and survives.

Worst possible outcome is probably OpenAI slowly goes under due to lack of talent and funding and become a MIRI-like, a few great people but with little resources and attention. Microsoft goes hard on e/acc and we all die at the hands of Altman and Satya.

Of course this all doesn’t account for competition dynamics with other companies which a whole separate nightmare to consider.

For now my p(doom) remains unchanged but I am 60/40 that I will update negatively in the next 3-6 months.

Nov 20, 2023
at
12:58 PM
Relevant people

Log in or sign up

Join the most interesting and insightful discussions.