Greg Meyer

Blog

Can Claude Build NYC?

NYC Heroes in a minute: super speed, web swinging, flight and a superhero landing, then the city from Midtown to Brooklyn, day into night.

City of Heroes

Long ago, when I was a kid, I loved playing MMOs. There was one that was really fun, not really because of the fighting tech, but because of the movement. The movement was really fun (at least for its time). And no game really captured that until Spider-Man did 14 years later.

My real curiosity as well was this: what can you build in a few days with modern tech? I did not want to create a finished game. I wanted to create a somewhat unpolished gameplay demo. A finished game would require a story and polished gameplay. But how much of a fairly small demo can you create?

So, when the whole discussion around AI games erupted with Tim Sweeney I started out asking: could I create a simple superhero game that let you move around the city as well as City of Heroes? But I quickly came up with a second question: could I put this in NYC?

The hero, in a teal suit and cape, flying over Central Park with Manhattan on both sides
Flying over Central Park.

Unreal and Godot

Because of Tim Sweeney I decided to start making it in Unreal, but I wanted to embed this in a webpage. I didn't want anyone to have to install this. (Installing something is great but this is gonna be at best a little demo, I'm not building a game I'm gonna sell.) Unfortunately, Unreal deprecated its HTML5 interface a few years ago. So I had to use Godot.

NYC

NYC is a very well documented city. Every building is documented in open data. There are tons of pictures of buildings. It should be possible to do. And I don't really wanna pay for the Google Maps data, I'd prefer to just build this up myself. So, what I did was I had Claude build out the first version of the movement tech, and in parallel to build out the city. We started with Manhattan. It found all the details from the city's open data and started building.

The buildings didn't look (on the facade) correct. They looked terrible. Some buildings were not correct. There were no trees. However, it was recognizably NYC.

We then used a dated 3D model that the city had put out. This got us closer, but still not quite there (in 2014, when it was released, the Edge was not yet completed and most of the supertalls didn't exist, we constructed those from other data).

Midtown Manhattan from Top of the Rock in late-afternoon light, the Empire State Building at the center
Looking south from Top of the Rock toward the Empire State Building.

There are approximately 1.08 million buildings in NYC. After Manhattan was finished, we built all of them. 300k of them have exact setbacks. For a smaller subset of "recognizable" landmarks (Empire State, Guggenheim, etc), we made their facades and shapes look realistic.

Rows of brownstones and street trees in Brooklyn, seen from above
Brownstone Brooklyn.
The Statue of Liberty on Liberty Island, with the Lower Manhattan skyline across the harbor
The Statue of Liberty, with Lower Manhattan behind her.

How it works

We used Godot to export to WebGL. We run this on Cloudflare. A Cloudflare Worker streams the tile data to the browser. The whole city is about 2GB of data compressed, 7GB uncompressed. We're not going to load that all in one go. We have to load it in sequence. So, the game engine guesses where you will be in 3 seconds and loads those tiles in.

We load in 3 different levels of detail. There's far away, where nearly the whole city is loaded if you are in Times Square or any other central location (this is only on the "high" graphics setting). This is so, wherever you are, you can see the skyline. There's close by, that's loaded in a lower level of detail. And then there's very close, that's loaded in full detail. But if you're flying at 150m/s, you have to guess where you will be next. Loading in just where you are now is not enough. So, we have the engine guess where you will be next and load that in.

The city seen from high above New York Harbor, the Manhattan skyline in the distance
From high over the harbor, the far level of detail keeps the whole skyline in view.

It Feels like NYC

It feels like NYC. There are cars and people everywhere. There are the sounds of the city. You can see the city and Times Square at night. It feels real and feels alive.

Times Square at night: crowds on the plaza under walls of glowing billboards
Times Square at night.
Midtown at night from above, with lit windows and the floodlit Empire State Building
Midtown at night: lit windows, a floodlit Empire State.
Bridges strung with lights across the East River at night
The East River bridges at night.

We added all the things you'd see in the city as well: trees, bike stands, bus shelters, LinkNYC terminals (Claude's idea!).

Street-level view up Fifth Avenue: taxis, buses, street trees and a church, with the Empire State Building behind
Up Fifth Avenue to the Empire State Building: taxis, buses, street lights and street trees.
A flyover: Midtown, Central Park, Brooklyn and Queens, from day into night.

Multiplayer

I wanted to make this multiplayer. Initially I thought we could do this p2p, with WebRTC. We needed a small signaling server for that. However, when I decided to create a public room we expanded the server to relay everything.

Signaling server: a Cloudflare Durable Object at /signal Matches players into rooms and hands out TURN servers Relays every packet of the public room introductions only every packet Player Player Player Private room, up to 8 Game traffic goes player to player Player Player Player Public room, up to 64 Every packet goes through the server
Co-op: signaling introduces players; only public-room packets pass through it.

Will you be a hero?

I thought this was a fun little experiment. I am shocked at how much you can do in about 3 days of lazy prompting. This took about 4 or 5 billion tokens. The future of gaming is very bright, not because people are gonna be making vibe coded games like this, but because you can build out a test of something in minutes.

It was well known that Starfield had issues internally before launch. But they had spent years building out this broken system. They couldn't abandon it! Veilguard was obviously a bunch of slapped together coop dungeons that were meant to be played as part of a Destiny 2 style MMO. It was not terrible, but it didn't work.

Games take so long to build and are so expensive you have to try to polish shit into gold. Starfield would have been a very different game if they could have created the first versions of this with Claude.

Games will end up being much more polished, you will no longer be committed to bad ideas. A bad idea can be thrown away. Check it out: nycheroes.party.