Image: marcopako
Hacking. Disinformation. Surveillance. CYBER is Motherboard's podcast and reporting on the dark underbelly of the internet.
Advertisement
Like other AI models including OpenAI's GPT-3, LLaMa is built on a massive collection of pieces of words, or “tokens.” From here, LLaMa can then take an input of words, and predict the next word to recursively generate more text, Meta explains in a blog post from February. LLaMa has multiple versions of different sizes, with LLaMa 65B and LLaMa 33B being trained on 1.4 trillion tokens. According to the LLaMA model card, the model was trained on datasets scraped from Wikipedia, books, academic papers from ArXiv, GitHub, Stack Exchange, and other sites.Do you know anything else about the LLaMa leak? Are you using it for any projects? We'd love to hear from you. Using a non-work phone or computer, you can contact Joseph Cox securely on Signal on +44 20 8133 5190, Wickr on josephcox, or email joseph.cox@vice.com.
Advertisement