- Emergent Behavior
- Posts
- EB-3: Building Intelligent Game Worlds
EB-3: Building Intelligent Game Worlds
Aincrad Here We Come
đź”· Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.
Follow
EB-3, our third podcast, dropped today. Before I continue, the rules of the game are:
Pods that CHART stays alive
Pods that get a Follow on Apple Podcasts CHART
Pods that get Listened to on any app CHART
So FIRST, CLICK on the link below (opens up your Apple Podcasts app), and click “+Follow” (in the upper right-hand corner)
Then go ahead and listen to the podcast any way you want to on your preferred app.
Listen on
Who
Vishnu Hari, CEO (LinkedIn), and Peggy Wang, CTO (LinkedIn), founders of Ego, an AI native game engine that is part of the Y Combinator Winter 2024 batch. Peggy + Vish are ex-Meta AI and Reality Labs engineers who are huge game nerds.
What
Ego is an AI native simulation engine and platform where users can create and share 3D animated characters, worlds, and scripted interactions all using natural language. Imagine being able to prompt the Matrix, Sims, or GTA into existence. They built 2 simulations (TownWorld and CharacterIdol) with their engine during YC and are backed by notable angels and funds such as Immad Akhund (Mercury), Kayvon Beykpour (Periscope), BoostVC, and PearVC.
When
Feb 22, 2024
Background
There are three big problems in open-world games:
It is hard for humans to write code and script interactions between assets, characters, and the environment to design fun games and experiences
It is impossible for NPCs to display human-like behavior and have true agency in these 3D worlds
It is hard to create new 3D assets, worlds, and non-playable characters
Ego has built an AI game simulation engine to solve the first two of these problems.
Avatars
But before that, Ego initially launched an iOS app called Ego that allowed users to become 3D-generated avatars with high-quality face tracking. Key points about the Ego Live Avatar product:
Users could scan their faces and use a character creator to customize their 3D avatars.
The app utilized face tracking technology without requiring a depth sensor, making it accessible on more devices.
Peggy, who previously worked on face tracking at Meta Reality Labs, brought expertise to improve the avatar's expressiveness and face tracking quality.
The app featured a universal rig, allowing users to switch between different avatar characters on the fly during livestreams.
Ego aimed to enable pseudonymous 3D social interactions through these expressive avatars.
The three avatar podcast
After launching the Ego app, they realized people cared more about avatar customization than face tracking quality and building customizations for avatars was not scalable at the time. As they were trying to figure out their next steps, they had a revelatory dinner with Emmett Shear, co-founder of Twitch (and former CEO of OpenAI).
Emmett advised them that what they were building downstream of 3D social spaces was actually a game, and they needed to build a game rather than just a chat app to succeed.
TownWorld
Around the same time, Joon Sung Park’s Stanford Generative Agents (aka Smallville), which simulated a small town using AI agents that could talk to each other such that they self-organized birthday parties and meetups, was released. Vish and Peggy were inspired by this to implement AI agents in 3D.
Smallville
This became TownWorld
1\ We built a 3D simulation of a small town populated by fully autonomous AI agents inspired by @joon_s_pk's Stanford Generative Agents paper. We called it TownWorld and live streamed it to Twitch
— Peggy Wang (@peggy_wang)
8:58 PM • Mar 12, 2024
TownWorld features autonomous AI agents populating a hand-crafted 3D map
AI generates all routines, conversations, and voices in real-time, while the team focuses on optimizing performance through multithreading and prompt optimization
A seven-day simulation with three agents costs only $7 in OpenAI credits, thanks to efficient resource management
The team implemented a "world marker" system to enable seamless navigation and object interaction within the virtual town.
Agents can navigate the environment, interact with objects, and keep track of their state and position
AI-generated voices powered by Eleven Labs enhance the immersive experience
Despite some quirks in the first version, the team has ambitious plans for future improvements, including added memory, more characters, and enhanced animations
Ego envisions an AI-native 3D simulation engine that empowers creators to generate entire virtual worlds using only natural language prompts.
AI will generate all components, from 3D maps and environment assets to character animations and interaction scripts
Creators can build games like TownWorld, The Sims, or Animal Crossing without any coding or animation skills
CharacterIdol
@character_idol #alieninvasion #alien #jefferythealien #radbotcomics #radbot #characteridol #funny #humor #meme #aigenerated
2 player dialogue game, where you can choose characters to converse with, built on the Ego framework.
Next up, the team presents at YC! Good luck, guys!
Transcript
Part 1
Part 2
Part 3
Links
In episode 3 of Emergent Behavior: @ego_ai_]co-founders @simulacronist and @peggy_wang join @8teAPi to dive into the world of 3D avatars and worlds.
--
TIMESTAMPS:
(00:00) Introduction and background
(08:09) Using the Ego app
(11:32) The vision for the game engine
(25:06)… twitter.com/i/web/status/1…— Turpentine (@TurpentineMedia)
1:09 PM • Mar 29, 2024
Become a subscriber for daily breakdowns of what’s happening in the AI world:
Reply