Bringing Llama 3 to life

Llama 3 is Meta’s most succesful openly-available LLM thus far and the recently-released Llama 3.1 will allow new workflows, equivalent to artificial knowledge era and mannequin distillation with unmatched flexibility, management, and state-of-the-art capabilities that rival the perfect closed supply fashions. 

At AI Infra @ Scale 2024, Meta engineers mentioned each step of how we constructed and introduced Llama 3 to life, from knowledge and coaching to inference. 

Joe Spisak, Product Director and Head of Generative AI Open Supply at Meta, talks concerning the historical past of Llama and Meta’s overarching imaginative and prescient for open supply AI.

He’s joined by Delia David, a software program engineer at Meta, to debate all issues data-related for GenAI. David covers the variety, quantity, and freshness of information wanted for GenAI and the way completely different knowledge varieties must be extracted and ready.

Kaushik Veeraraghavan, a software program engineer at Meta, discusses how Meta trains Llama at scale and delves into the information middle, networking, and software program investments which have enabled the event of Meta’s Llama 3 fashions.

Lastly, Ye (Charlotte) Qia, a manufacturing engineer at Meta, discusses how Meta handles inference for Llama. Optimizing and scaling LLM inference is vital for enabling large-scale product purposes. Qia introduces key parallelism strategies that assist scale mannequin sizes and context home windows, which in flip affect inference system designs. She additionally discusses sensible challenges related to deploying these advanced serving paradigms all through Meta’s inner cloud to our knowledge middle of heterogeneous {hardware}.