1,732 subscribers
Gå frakoblet med Player FM -appen!
Podcaster verdt å lytte til
SPONSET
Genie: Generative Interactive Environments with Ashley Edwards - #696
Manage episode 432663114 series 2355587
Today, we're joined by Ashley Edwards, a member of technical staff at Runway, to discuss Genie: Generative Interactive Environments, a system for creating ‘playable’ video environments for training deep reinforcement learning (RL) agents at scale in a completely unsupervised manner. We explore the motivations behind Genie, the challenges of data acquisition for RL, and Genie’s capability to learn world models from videos without explicit action data, enabling seamless interaction and frame prediction. Ashley walks us through Genie’s core components—the latent action model, video tokenizer, and dynamics model—and explains how these elements collaborate to predict future frames in video sequences. We discuss the model architecture, training strategies, benchmarks used, as well as the application of spatiotemporal transformers and the MaskGIT techniques used for efficient token prediction and representation. Finally, we touched on Genie’s practical implications, its comparison to other video generation models like “Sora,” and potential future directions in video generation and diffusion models.
The complete show notes for this episode can be found at https://twimlai.com/go/696.
733 episoder
Genie: Generative Interactive Environments with Ashley Edwards - #696
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Manage episode 432663114 series 2355587
Today, we're joined by Ashley Edwards, a member of technical staff at Runway, to discuss Genie: Generative Interactive Environments, a system for creating ‘playable’ video environments for training deep reinforcement learning (RL) agents at scale in a completely unsupervised manner. We explore the motivations behind Genie, the challenges of data acquisition for RL, and Genie’s capability to learn world models from videos without explicit action data, enabling seamless interaction and frame prediction. Ashley walks us through Genie’s core components—the latent action model, video tokenizer, and dynamics model—and explains how these elements collaborate to predict future frames in video sequences. We discuss the model architecture, training strategies, benchmarks used, as well as the application of spatiotemporal transformers and the MaskGIT techniques used for efficient token prediction and representation. Finally, we touched on Genie’s practical implications, its comparison to other video generation models like “Sora,” and potential future directions in video generation and diffusion models.
The complete show notes for this episode can be found at https://twimlai.com/go/696.
733 episoder
Tutti gli episodi
×1 Evolving MLOps Platforms for Generative AI and Agents with Abhijit Bose - #714 58:08
1 Why Agents Are Stupid & What We Can Do About It with Dan Jeffries - #713 1:08:49
1 Automated Reasoning to Prevent LLM Hallucination with Byron Cook - #712 56:48
1 AI at the Edge: Qualcomm AI Research at NeurIPS 2024 with Arash Behboodi - #711 54:47
1 AI for Network Management with Shirley Wu - #710 53:44
1 Why Your RAG System Is Broken, and How to Fix It with Jason Liu - #709 58:03
1 An Agentic Mixture of Experts for DevOps with Sunil Mallya - #708 1:15:09
1 Building AI Voice Agents with Scott Stephenson - #707 1:01:44
1 Is Artificial Superintelligence Imminent? with Tim Rocktäschel - #706 55:52
1 ML Models for Safety-Critical Systems with Lucas García - #705 1:16:06
1 AI Agents: Substance or Snake Oil with Arvind Narayanan - #704 54:22
1 AI Agents for Data Analysis with Shreya Shankar - #703 48:24
1 Stealing Part of a Production Language Model with Nicholas Carlini - #702 1:03:30
1 Supercharging Developer Productivity with ChatGPT and Claude with Simon Willison - #701 1:14:15
1 Automated Design of Agentic Systems with Shengran Hu - #700 59:30
Velkommen til Player FM!
Player FM scanner netter for høykvalitets podcaster som du kan nyte nå. Det er den beste podcastappen og fungerer på Android, iPhone og internett. Registrer deg for å synkronisere abonnement på flere enheter.