Coffee Shop Thoughts on AI, LLMs, and Robotics

It has been way too long since I’ve written. Last post I wrote was like 6 months ago. Life has been busy!

I’ve got <1 hour at a coffee shop so I’m just going to brain dump as much as I can on this topic. Let’s get after it.

From my vantage point, it’s pretty clear that we’re going through the early stages of an incredible emerging technology. There’s definitely a little too much hype right now and most folks are still trying to find product-market fit in the face of a technology that is incredibly agile. Agile in the sense that it can commoditize entire workflows and “moats” very quickly. Startups are being forced back into the gym and get in shape in terms of speed, flexibility, agility, and market positioning.

The market seems to be trending in a couple of different ways. The first is that I think we’ll start to hit diminishing returns on what we can train LLMs on from public data. The “macro model” war feels a lot like the search war where there were a lot of players in the early stages but only a couple of real winners have successfully commercialized and gained market share (eg. OpenAI & Anthropic).

This is interesting because it implies that where folks will gain alpha with these models is actually in the opposite direction: small models on proprietary data. This is the second trend. I’m not the only one who thinks this as Mark Zuckerberg was on a podcast recently (can’t remember which) and said something very similar. Lower count of parameters but insane quality from smaller datasets.

We’ll get generalized AI from the public, large models but get well-tuned AI that can commoditize specific skills, workflows, etc. on an industry-by-industry basis. Both will be required in this transformation.

I believe the market is generally trending towards this and it shows in the battle between vanity vs. value. With large generalized models, they’re great for really high-level understanding and knowledge smithing. However, trying to jam them into certain workflows, environments, etc. continues to produce lackluster results. This is only possible with really high-quality smaller data sets that have fine-tuned models. Most of that data right now that can actually generate “value” is locked away in data silos and are proprietary.

This is part of the current limiting factor on these models. There’s just not enough high-quality data on specific enough things to really produce value. The other end of the limiting factor spectrum is access to compute. It’s still hella expensive to train a model. It’s hard to get the level of compute required to do it well and so we are seeing a natural war of “who has more cash”.

I love seeing this because this is the type of environment that precedes radical innovation. When markets become incredibly constrained (but not high regulated), it pushes for new innovation to emerge.

That’s neuromorphic and thermodynamic computing. Current GPU architectures are great at handling deterministic numbers. However, LLMs are not deterministic but rather probabilistic. To fully scale LLMs, we need a full-stack architecture (hardware and software) that operates off of probabilistic modeling and unstructured data.

There are companies working on this that, quite literally, could reduce compute costs by 100x and increase performance by 100x simultaneously. For the last ~70 years, we’ve been building our current computing knowledge and architecture based on the initial inception of the computer concept. However, we now need to fundamentally rethink the entire stack in order to keep up with Moore’s Law on the growth and value of LLMs.

Extropic, Normal Computing, and Rain AI (among others) are all working on this problem, and in my eyes, they have the potential to absolutely nuke Nvidia. So, for the first time in my life, I’m actually sort of bearish on Nvidia and bullish on startups.

Bubbles can pop in different ways.

On the topic of probabilistics, I can think of no better industry that needs this than Robotics.

Think about the process and dexterity required to fold laundry. There’s millions of micromovements that are required in order to fold laundry and they change based on a ton of different parameters. Example: clothing size, clothing type, texture, folding style, etc.

When folding laundry, your arms and hands move in specifically directed but imprecise ways. An example of this is grabbing the edges of a t-shirt. My right hand my pinch on the shoulders in the exact place I expected with good grip but the left hand may pinch but slip slightly as I pick up the t-shirt because of the weight. It’s the small things.

While some could argue that yes, each of these states by second has a discrete value (eg. x, y, z position) and so deterministic works here, the challenge still comes down to how you train the model on this discrete data set. I’d argue that this is a very unstructured process that produces very unstructured results/data which drives the need for a higher range of errors.

Anyway, that’s a long-winded way of saying that there is going to be a much deeper convergence of the current path of AI and Robotics. The small parameter models on high-quality data is what will give these robots the humanistic ability that we have all been sold on from sci-fi movies.

C3PO ain’t that far off. Like, V1 by 2034…

So, where does this all head?

Honestly, I’m not sure. It’s easy to say stuff like “LLMs will continue to get better” which, like, is obvious but not helpful at all.

Going back in history a little bit, when I first started my career, I was doing some basic market research and found that widely used stat that said something like “70% of the companies on the S&P500 list from 1970 are no longer on it”. The reason was that they either adopted technology, or they died. It was that simple. The age of the internet has arrived and provided a way to transform every industry. Those who embraced it thrived substantially while those who scoffed and fought it died.

It’s my opinion that we’re going through this cycle once again. Those who understand how to capture unique, high-value data and then create a valuable model on top of it will thrive. They will thrive because they will be utilizing this new emerging tech as a weapon to capture market share. Companies that have been single-product focused can all of a sudden move into entirely new product areas with significant disrupting force, simply because they have no priors.

You either embrace it, understand it, and weaponize it against the market, or you die. Whose most at risk? Small businesses. Who has the biggest opportunity? Small businesses.

What’s funny is that this really does feel like the 2000’s Dotcom bubble all over again but in a very different way/perspective. Up until around 2021, there was just a huge concentration at the very top of “who owned what”. Google owned Search. Amazon owned e-commerce. Salesforce owned CRM. You get the picture.

And then, all of a sudden, the bubble popped. I’m not sure why but it really does feel like after Musk bought Twitter, that was the moment everything changed. Perhaps because ideas could now flow freely without scrutiny. I’m not sure. But, things have been completely different since then and the innovation has reared its head.

If Musk buying Twitter was the demarcation point then we are going through a “deflating the bubble” period. Bloated organizations are trimming the headcount to reduce internal political battles that create unnecessary friction (as well as improve balance sheets). SMBs and startups are operating super lean and focusing on profitability, but the key is that they’re doing more with less.

I personally know of multiple companies with <15 employees growing at $1M/mo simply by exploiting the bloat of a larger organization.

I’m looking at you, Salesforce and SAP. If you work for either of these companies and your CEO isn’t shitting themselves, then I’d cash in your stock and find someone is giving a shit about it.

Remember, things happen slowly, then all at once. Escalator up. Elevator down.

That’s all I’ve got for today. I’ll close with this thought:

We’re at a really interesting time in history. Things are no longer linear but exponential. Things are the most “tight” and concentrated they have ever been, both within this sector as well as the broader economy. We have all of that paired with a highly lubricated financial and telecom infrastructure where I can move capital and data on a dime.

It’s a ripe environment for a black swan event.