Tools are enablers. With tools, people build cool things. Let’s give people more tools!
My buddy Chris edited this vintage of Unbeaten Path with these world-beating D&B kings.
The coolest part of my job happens everyday. I get to speak with people that are smarter than me. A few days ago, I was discussing chip architectures with a founder and he said:
“Think about it: how many things are we not trying to do, just because trying is so expensive?”
This is triggering. Picture yourself as a 50-year-old in a dead-end job and you overhear this. The voice in your head would urge you to call your boss and quit. Immediately. For me, I start reading about technology, fall into a rabbit hole, and end up writing a post to convince myself that I haven’t wasted my time.
Cost of trying in AI
We all know this graph: training LLMs is just so expensive.
There is, however, something I never considered when looking at this: these are the costs of training the final iteration of each model. How many training exercises did OpenAI perform before fixing the architecture and all the tiny details within GTP-4? I suck at writing software, and if I had to pay for every time I tried to compile my code and failed because I forgot a semicolon, I’d be broke in the first week of my job.
When examining the problem through this lens, you start wondering: how many smart people aren’t trying to build next-gen super-duper-mega AI model, just because they don’t have the hardware to support their very smart ideas (and they don’t work for OpenAI)?
Let’s not forget that AlexNet literally destroyed the competition with an architecture that was previously too hard to run. They could do it also because they used a different kind of hardware - GPUs.
The lever to move the world
Think about open-source software. Almost all of the world runs on Linux, an open-source project. It's impressive what a few geeks can build in their home offices.
Talking about home offices: there is a video on the web of Linus Torvald touring his very own. For those of you who don’t know him, Linus is the creator of Linux.
There is one thing that caught my eye in the video: Linus is a SW guy, maybe the king of SW guys, and his desk is full of… hardware.
This brings me to the point: hardware enables innovation.
There is no Divina Commedia without the pen Dante used to write it, there is no YouTube without the hardware to support the required bandwidth, there is no Substack and no Unbeaten Path without the Intel Core i7 running in my laptop.
But once there are pens, there are kids in every corner of the world that can write something powerful every day. With YouTube, anyone can record and upload a song. Once there is a powerful processor, every geek can open the laptop and start coding.
Not every kid will become Dante Alighieri, not every amateur singer will become Justin Bieber and not every geek will invent Linux, but maybe someone will!
This is what hardware does: it hands a lever to enough people, and maybe the right person will activate it and move the world!
The case for Hardware
There are two pieces of writing that I read recently - The Bitter Lesson by Rich Sutton and How to Predict the Future by William Hertling. If you haven’t read them, I suggest you do.
Both essays give very brilliant examples of why investing in hardware is a good idea.
The concept is pretty simple: between human intelligence, and the tools human intelligence has to express itself, the bottlenecks will always be in the tools.
Give the smartest people an underperforming tool, and they won’t achieve much. But if instead you build the right tools and release them into the wild, the intelligence of the crowd will find a way to use them to make good things.
Let’s build these tools, then!