For the life of me, I've been trying to find a New York-style bagel. For those of you who may be unaware, I'm in the Midwest (Milwaukee, to be precise), and our bagel game is nowhere near New York's. There's a few trying to contend for it, but it's like trying to shoot a basketball from your backyard to a hoop on the moon: astronomically unlikely. There's plenty of theories: it's the water, market pressure makes things good, etc. - I know one thing is outside of theory, however, and it's that I'd commit war crimes for a good Cuban on a bagel.
Anyways, speaking of astronomical odds, the fascinating topic of the week is data, and data centers. It's about as controversial a topic as you can get (next to asking a New Yorker what the best bagel place in the city is). Musk and SpaceX plan on orbital data centers, and the ones on the ground are equally as inflammatory.
That being said, I'm going to shed a little light (as an engineer) on the nature of these, briefly. Why the hell do we need data centers anyway in the first place? Well, the phone or site you're reading this on runs off a data center. Every single bit and byte that gets transmitted needs some kind of data processor for it. The recipe for bread hasn't changed for 10,000 years, and this rule hasn't really changed for the past 40 with the Internet.
With the advent of AI, the demand for data processing became astronomical: large language models, image models, voice models, and so on - if you have used Siri, Alexa, ChatGPT, Google, its probably likely you've used (or invoked) some combination of language, image, and voice models. What these models are is essentially harnessed statistics - given X input, what is Y output? Not unlike the way our brains work. What's the most likely thing that comes to mind when I say "bagel"? Or "bread"?
Probabilities of certain words being returned by language models during inference
This is an example of language tokens (snippets of language - ordered by probability of being returned by the language statistical model). What all these data centers do, partially, is invoke these probabilities countless times per day for any arbitrary purpose. Someone might be writing a business proposal. Others could be generating spicy fan fiction. Someone could be chatting with a voice that sounds like a long-lost friend. A researcher could be generating autonomous pathways for novel proteins to cure a specific form of cancer.
That last one is real - a tech founder (who is also a data analyst) in Australia created a personalized cancer vaccine for his dying dog with the help of ChatGPT and AlphaFold (for protein analysis). This is incredibly recent news - and we're only going to see more.
The point is, these data centers do a lot of heavy lifting - and they're spread out everywhere. People have sometimes asked me: "can't we put these all in the desert and power them off solar, instead of displacing whole communities with them?" - and there's not a clean answer to that. One of my best rationalizations is this:
Imagine you're in a self-driving car. In order to detect and avoid potential collisions, this car needs to constantly examine the cars around it and find how to maneuver in a way that best preserves you, the occupant. To do this, the car needs both onboard processing and communication to a nearest node (i.e. data center) to have as quick a response time as possible.
The threshold for something to seem instantaneous to a human is roughly 100ms - for a signal to round trip reach San Francisco to New York, and then your car wirelessly through a cellular relay, is way too unforgiving (and downright impossible under "instantaneous" circumstances). There's a funny story about a person being unable to send emails more than 500 miles because their router would time out immediately - governed by the speed of light. Data centers being spread out the way they are is partly a result of the natural bottleneck from transmission technologies.
So piecing this together, we can begin to see the rationale for them the way they are now - and for space data centers. When I develop applications, I usually have a choice of servers and data centers I can host them on - us-east-1, us-west-1, etc., or Europe, or Asia, or anywhere else that is closest to my user. With satellite transmissions (and novel material science technologies in microgravity plus orbital solar power), we start to get a lot more efficient with how we can process and transfer all those bits and bytes.
If I had my way, I'd spread out (or at least copy-paste) every bagelry in New York across the country to give everyone a chance at a proper bagel at any time. Rather than serving bits and bytes, they serve deliciously tasty sandwiches on freshly baked fluffy bagels that rarely fail to please. If you couldn't tell, I wrote this musing while I was thinking about bagels. And about the nature of distributed data, the need for compute, and root causes.
Let me know what flavor of this you enjoyed, and as always,
Be well,
Michael Kirsanov