Aviv Yovel (@avivyovel): "We keep treating LLMs like a garbage bin, throwing more and more data in and hoping something magical will happen.But just like people, these models have a short attention span. The more noise you add, the worse they perform. Smarter context, better results. thanks Chroma for a…"

Make money doing the work you believe in

We keep treating LLMs like a garbage bin, throwing more and more data in and hoping something magical will happen.But just like people, these models have a short attention span. The more noise you add, the worse they perform. Smarter context, better results.

thanks linkedin.com/company/tr… for a great piece of research, really worth a read:

trychroma.com

Context Rot: How Increasing Input Tokens Impacts LLM Performance

Large Language Models (LLMs) are typically presumed to process context uniformly—that is, the model should handle the 10,000th token just as reliably as the 100th. However, in practice, this assumption does not hold. We observe that model performance varies significantly as input length changes, eve…

Nov 5

2:24 PM

Make money doing the work you believe in

Log in or sign up