Everyone needs to take into account the costs behind LLM-generated video. The challenge is that LLMs can’t seem to get costs under control. An LLM’s cost is a relatively simple formula: cost per token multiplied by the number of tokens required. The good news is inference costs seem (again, we have terrible data on it) to be coming down. Say 5% per year. The bad news? To improve models, the companies need to drastically increase the number of tokens used per answer. But not in percentage terms, but orders of magnitude. So instead of using 1,000 tokens, new answers take 10,000 tokens.

Do that quick math. (Just for illustration.) Say a million tokens cost $1, and to answer most questions, it requires the model to spend 10 million tokens. (It doesn’t, I rounded, but go with it.) So, $10. Then assume costs go down to $0.095 per million. Great, 5% cost reduction! But the model requires an order of magnitude more tokens, so you use a billion tokens. That’s $95! A 9.5X increase. Thus, while the costs of inference head down, the total LLM costs go up. Based on Ed Zitron’s reporting, I think that’s happening.