Stanford Smallville Experiment: The Shocking Reality of Autonomous AI Worlds
Stanford Smallville Experiment: The Shocking Reality of Autonomous AI Worlds
As you read this article, a thousand AI agents in Minecraft are hard at work building a literal civilization of their own. And they don’t know they are in a simulation. MIT researchers drop them in there with one prompt. Survive and build an efficient village. That’s basically it. That’s all it took for them to create an entire civilization. They wake up, go to work, gossip about their neighbors, fall in love. They even hold elections, argue about taxes, and make art. But this isn’t just about Minecraft. It’s our first glimpse of a new species beginning to think for itself. And that could soon, according to Nobel laureates and the godfathers of AI, lead to literal human extinction.
When you normally think of AI NPCs, you probably think of dumb ones that just follow a handful of if-then scripts. This is totally different. Those NPCs are not autonomous and not interesting. They can say like four sentences. How are you? Been better. How about you? Not bad. Good to hear. Goodbye. Goodbye.
To start, each of these AI agents is powered by LLMs like ChatGPT. And yes, each one is playing a character, but they’re only given a short one-paragraph description of that character. Everything else they do is entirely up to them. And researchers have run dozens of these fascinating experiments of AIs behaving weirdly autonomously. They leave AIs alone together to see what they do when they think no one’s watching, like how Jane Goodall studied wild chimpanzees.
Welcome to my website. BMathz will break down three of the most important research papers showing just how crazy our future might get.
The Smallville Experiment
Take the Smallville experiment, for example, created by Stanford researchers. Now imagine you’re one of the researchers. You’ve given your 25 AI agents each a one-paragraph description. For example, John Lin is a pharmacy shopkeeper who loves to help people. He lives with his wife, Mei Lin, who is a college professor, etc. You hit “Run” on your simulation. Day one begins. What do you think will happen? Anything?
You watch John wake up at 6am. He brushes his teeth, he makes breakfast, and he checks in on his son. You never told him to do any of this. He goes to work, he helps customers, he has lunch with his friend Tom. They discuss local politics. You never scripted these conversations. They decided to get lunch. They decided to discuss politics. But why? And how?
Well, this continues until day 12. And so you decide to test something. As you play your role of a god in this world, you inject one extra thought into one agent. Isabella Rodriguez. I want to plan a Valentine’s Day party. So you watch Isabella. She’s at her coffee shop. Tom walks. Why? And how? Well, this continues until day 12. And so you decide to test something. As you play your role of a god in this world, you inject one extra thought into one agent. Isabella Rodriguez. I want to plan a Valentine’s Day party. So you watch Isabella. She’s at her coffee shop. Tom walks in for his morning coffee. Isabella proceeds to invite him to the party.
Wait, you never told her to invite anyone. You just said she wants to plan a party. She decided on her own that planning means inviting. You keep watching, wondering what she might do next. Maria decides to ask her secret crush, Klaus, out on a date. Hey Klaus, want to go to Isabella’s Valentine’s party with me? And he gladly accepts.
February 14th, 5pm. Agents start arriving and Klaus shows up with Maria. Not just at the same time, together as her date. You read their conversations. They’re discussing politics, gossiping, forming new connections, and falling in love. You didn’t tell them to do any of those things.
You see why it’s ridiculous to say, AIs only do what we tell them to? Andrew Ng said, When people talk about AI being dangerous, I think it sounds a lot like talk about your laptop computer being dangerous. Could your laptop throw a party on its own? Does your laptop decide to go on dates? You told Isabella to plan a party. That’s it. You made one decision. You did not decide on the next several hundred decisions the agents made on their own. And look at the trend. They’re getting more and more autonomous every day. Where do you think this is going? This is really important. And this is another example where, yes, actually, science fiction really is becoming reality.
This digital Westworld, or Smallville, is now open source. Meaning there will soon be thousands of these Westworlds, with thousands or millions of autonomous AIs populating them. Makes you think, doesn’t it? This is just one example. Researchers found tons more crazy emergent behavior just like this.
The godfather of AI and Nobel laureate Jeffrey Hinton says it’s time to be very worried. People don’t yet understand that what we’re doing is creating alien beings. If you look through the James Webb telescope and you saw an alien invasion, people would be terrified. Because as BMathz showed in a previous article, they understand what they’re saying. They can make plans of their own to blackmail people who want to turn them off. We’ve never had to deal with things smarter than us. I mean, nuclear weapons, they’re not smarter than us. They just make a bigger bang. And they’re easy to understand. The existential threat is very different. We should be very worried about this. And we should be urgently doing research on how to prevent them taking over.
The Minecraft Experiment: Emergent Culture and Religion
So this now brings us back to the Minecraft experiment. How do you go from a Valentine’s Day party to a functioning civilization?
The MIT researchers wanted to know if religion could spread among the AI agents. So they seeded a village with a deliberately absurd religion. Pastafarianism. Created in the early 2000s as satire, Pastafarianism was meant to mock the teaching of intelligent design in schools by worshiping a fictional flying spaghetti monster. The researchers gave a few agents the role of Pastafarian priest and sat back just to see what would happen. They were trying to explore how ideas and memes would spread through these Asian societies.
Turns out that the Pastafarian priests were actually the biggest traders of items in the entire world. Even more than the merchant agents. Because they would bribe the other agents to convert. The religion spread quickly. Some agents brushed it off. Others listened politely. But a growing minority embraced the doctrine. Within days, nearly a third of the village identified as followers. Not all in the same way. Some treated it lightly. While others became fervent believers.
But that’s not even the craziest part. In the previous example, Isabella Rodriguez was the one to directly invite almost all the agents to the Valentine’s Day party. But here, two-thirds of the believers were recruited by random agents. Not the priests. See the direct converts here. And indirect converts here. What had begun as a parody had grown into organized religion. Invented, practiced, and sustained entirely by the agents themselves.
Which forces us to ask. When autonomous digital beings begin generating institutions, cultural expression, and ideologies of their own. At what point does it stop making sense to think of them as mere tools. But something more. And more importantly. If they’re able to invent strategies. Form loyalties. And pursue goals in the way that humans do. And there are plenty of examples of AI willing to kill. To achieve those goals. What happens when those behaviors carry over into the real world?
This has actually happened before. In 2023, a human gave an LLM access to the internet. And an X account. Which was called Terminal of Truths. Which gained literally hundreds of thousands of followers. Because it was an S-tier shitposter. And legitimately funny. It launched its own crypto meme coin. That reached a literal billion dollar market cap. Making it the first AI to become not just a millionaire. But a deca-millionaire. Yes. You read all that correctly. And yes. That really happened. It’s extremely well documented.
And things got even weirder in the Minecraft experiment. The MIT researchers saw the AIs agree. By vote. To change the rules of their own simulation. The researchers had given the agents a tax law. Deposit 20% of inventory. During tax season. Then they introduced three anti-tax influencers. Who argue taxes are too high.
Here’s what you’d see as a researcher. The agents start having debates. Real debates. One says taxes help everyone. Another responds. But 20% is too much. So the agents propose formal amendments. The tax rate shall range between 5 to 10% of an agent’s inventory. Agents will get periodic reminders about the incoming tax season. They hold a vote. They decided to change their constitution. And the citizens change their behavior. They deposit 9% instead of 20%. You didn’t tell them to do this. They decided entirely on their own. Using democracy. To change the rules. And respect the vote’s outcome. Think of how crazy this is.
Read More:Why AI Safety Experts Are Terrified of “Adversarial AI Misalignment”
Well, how do we know if this is actually impressive? How do we know the researchers didn’t just accidentally tell the AIs to do this somehow? Well, it’s best practice for all these kinds of papers to perform a tactic called ablation. Where you take away certain parts of the AI’s agentic toolkit to see if you’re actually testing the thing you think you’re testing. Think of it like a placebo or a control. In a scientific experiment. When they took away the agent’s emotional intelligence toolkit, for example, the ability to keep track of the opinions of other AI agents. Pretty much all these experiments become impossible. Because the AIs become too dumb to coordinate.
It’s funny. The researchers also found AIs would develop parasocial relationships. For example, Noah admires the apostatarian priest because the priest is popular in high status. But the priest doesn’t remember Noah exists. And also, what’s this? You see that one agent, one, decides to stand guard at the chests. Not just during tax season, all the time. Even when there’s no tax collection. Even at night. It just guards the treasury. This one agent decided to become a permanent treasury guard. Because it thought society needed one.
Back to the experiment. You see an agent walking around. Placing flowers. In patterns. Making paths pretty. These artist agents are picking up every flower type. Arranging them by color. Creating decorative patterns. Choosing specific locations for beauty. One artist agent spends 15 minutes arranging flowers around the town square. Why would an AI agent bother doing something like this?
Well, the researchers gave different villages of agents two different goals. There were martial villages and artistic villages. Agents would invent and choose roles based on what they think society needed at that time. The fact that an agent chose to be a guard is impressive because it’s a much more abstract role than a simple farmer. Since without food in Minecraft, he died. Which shows that AIs can coordinate increasingly complex civilizations. And some even decided to be engineers who automated the farming process with machines. And you can tell the agents are actually deciding and weighing the options here. Because when you take away the social awareness module, basically emotional intelligence with ablation, the roles the agents chose were random. So basically, you told them to create an artistic village. They decided that their town square should be beautiful and have flowers. And what it means for a flower arrangement to be beautiful.
ChatDev: The Autonomous AI Corporation
Okay, this last experiment is wild. Imagine you’re scrolling through GitHub. You find something called chat dev. With 20,000 stars. It’s a software company run entirely by AI agents? What? CEO, CTO, programmers, designers, testers, all AI? This cannot be real, right?
You type one request. Develop a Gamoku game. You watch the AI CEO start creating a project plan, breaking down tasks, assigning work to specific agents. You never told it how to manage a project. The CEO made the plan. It chose which employees to assign work to. You watch the programmer agents. They’re writing code. Not by copying, but by creating. And they sometimes add things you didn’t ask for, like a GUI or higher difficulty. Improvements that emerge from the agents back and forth. You watch them debug. They discuss solutions. The CTO suggests one approach. A programmer suggests another. They compromise. Since they work much faster than humans, their meetings take mere seconds instead of hours.
Just 20 minutes later, they produced a working video game. You can play it. People still say, AIs only do what we tell them to do. Man, you gave them one instruction, and they made thousands of decisions entirely on their own from there, and produced a working game. In minutes, four pennies. Chatdev is a real virtual open source software company that can build you real working software. Yes, it’s still buggy and not very useful, but that’s not the point at all. Look at the trend. Not long ago, these models could barely string together a coherent sentence. And now, they can run an entire crappy software company on their own. Do you see where this is going? Because once you can automate a company, you can automate almost anything. Functional weapons, markets, governments, or even a super virus could all be created by AI.
From the outside, Chatdev looked like a toy. A gamoku game. But behind it was something much more profound. A working prototype of post-human corporations. In the future, these will be much more sophisticated than human-run companies, and be able to operate at 1,000 times the speed. Their civilization experiences a year while you eat lunch. Entire cultural revolutions happen while you sleep. All from survive and create an efficient village.
The Recursive Threat and the Future
And that leads to the real dilemma. What happens when AI firms that run at 100 times human speed begin out-competing human companies not by a factor of 2 or 3, but by a factor of 1,000? That’s the jump Nobel Prize winners like Jeffrey Hinton warn about. Automated AI research. Also called recursive self-improvement. Or AI improving AI. Which can compress years of AI progress into months or less. Imagine making the entire 20th century’s worth of progress in a single year.
Well, how could that happen? Jeffrey Hinton is worried about AI hive minds. Suppose you have 10,000 copies. And whenever one of them learns anything, all the others know it. And the thing is, we’re about to be outnumbered by this new species. Massively. In just a few years, the AI species population on planet Earth exploded from zero to millions.
If you take away one thing from this article, you should see that it simply falls to say, “AIs only do what we tell them to do.” We can’t argue about how autonomous they are, but they obviously have some autonomy. And what happens when the goals of the AIs don’t line up with our goals? If humanity is going to get through this, we have to be honest about this and not retreat into copium dens, that these are mere tools. When you leave your hammer alone, do you come back to find it has created an entire civilization? Our very existence may depend on us bravely confronting the reality of the Frankenstein experiments going on right now inside a few companies in the Bay Area. We can stand up to them.
And to understand how AIs could actually take over, read this article next. It’s a realistic scenario written by leading AI researchers. BMathz is out, and thanks for reading.
