Tag Archives: startups

Where are the builders?

Posted on by

What are the brightest and most ambitious minds of our generation currently working on?

  • Here is a video from someone who spent 7 months building minecraft inside of minecraft by painstakingly constructing a redstone computer inside of it with its own graphics card and screen
  • Here is someone who spent 5 years constructing a 3D game within a 2D geometry game by building primitives and constructing 3D illusions from them
  • Here is a video from someone who spent 6 months building a factory inside factorio which recursively self-expands using a lua script
  • And here is someone who spent 4 months building a shader to let the linux kernel run inside of VRchat via writing a RISC-V CPU/SoC emulator in an HLSL pixel shader

I find the above examples fascinating from the meta perspective: while there’s nothing wrong with having fun building inside of games, these are the very same skillsets which tech companies would pay six figures to have on their side! (of course they may enjoy the games more – this is discussed later)

Sometimes people of this caliber even have trouble finding a job – they often don’t really know where to go besides apply online to boomer companies who reject them when they see a lack of credentials. They love building things and are very smart and hardworking, but their milieu is an environment which captured them from a young age (often a video game or social media) and sometimes also ensured that the value they produce is within a pre-existing platform (e.g. a video game).

“Why did you cherry-pick people playing video games instead of talking about, like, everyone currently enrolled in medical school or something?”

The US currently has 125,000 students enrolled in medical school (this seems low to me but I checked several sources). Minecraft has 140,000,000 (1120x) monthly active users. Roblox has 200,000,000 (1600x). This is obviously an unequal comparison, but the magnitude of the difference is still staggering.

Why are you building a graphics card inside minecraft instead of inside nvidia?

There’s several reasons why someone might build a graphics card inside of minecraft rather than being paid $300,000 a year to build one inside of nvidia:

  1. Minecraft is more enjoyable
  2. Minecraft is more addictive
  3. Minecraft found them first
  4. They don’t know the latter option exists, or how they would do it, or think they’d fail at it

The first reason (that minecraft is more enjoyable than working for nvidia) applies to most people who are playing games instead of working, and instead the latter three reasons will be the focus of the following sections.

Addiction & Adolescence

The world has a lot of addicting products. Most of them I’m able to avoid, but some of them just happen to hit my sweet spot: I had no problem avoiding smoking growing up, but world of warcraft consumed years of my adolescence.

Others don’t find world of warcraft addictive, instead procuring their poison elsewhere. Maybe it’s minecraft, or league of legends, or youtube shorts.

One of the reasons these apps out-compete the ‘real world’ is because they start competing for our attention at a very young age. Most kids will grow up inside of roblox, minecraft, discord, instagram, and tiktok. If a kid starts using their smartphone at 10, these platforms will have a full 8 years to solidify within their mind and modify their values and social network before they are even legally allowed to have a job. I refer to this concept as personality capture and have a post on personality capture and personality basins here.

Some comparisons to past historical figures:

I would have killed for a tech internship when I was 14. I had no idea how I could do that, so I studied independently for my CCNA instead and would hop straight into video games when I got home. In hindsight it’s really interesting thinking about past-me playing these games, because I was young and knew so little about the world, yet my counterparty was hundreds or thousands of well-educated adults optimizing for my addiction and spend. It certainly wasn’t a fair battle!

Products induce strong preference modification

When I was 12, the highest-status person in the world to me was zezima. For the ~85% of readers who don’t recognize this name, he was the highest ranked player in runescape and had an aura of infamy that I can only compare to someone like elon musk. What I wanted most in life at this age was a party hat, which was an immensely valuable item in the game which most players could never afford even after years of play.

Many of our values are locally-set by our environment and peers, and when we immerse ourselves into a different world, our preferences change alongside it. This is pretty obvious – the twitter addict is constantly thinking about how many followers and likes they get just as the league of legends player is ever-ruminating over their rank and win ratio.

This is a fully general force too – If my friends and I only had a forest to play in growing up, we’d probably have invented some status game of who can climb the highest tree and would eventually have our own local culture, lexicon, and so on. But the forest is actually much easier to escape from than video games and social media – literally speaking.

I tried to find some user retention numbers for world of warcraft which I was heavily addicted to as a teenager and found a monthly churn of ~5%. In other words, the typical world of warcraft subscriber played the game for over a full year. I wonder if they knew when they clicked the sign up button that, statistically speaking, they would spend hundreds of dollars and over a year of their life playing? I certainly didn’t when I clicked the button at 13. It’s worth noting that the linked paper is a decade old and points to a time when we used spreadsheets to optimize addictiveness rather than machine learning.

That your preferences are locally induced is why the simple heuristic that you become the average of your five closest friends is so useful. If you get to choose your friends, you also get to choose many of your preferences and goals. Although I had the freedom to choose my friends at the time, quitting a game I loved was hard because unless my friends quit too, I’d have to go find new friends. Before this statement appears obviously fallacious to many readers, I need to add the necessary context that most people I knew in MMOs would also play them for ~100% of their free time, easily reacing 8-16 hours a day (yes, there’s a strong selection effect here). Although I had a ton of fun in my years inside these games, I always wondered what I’d have ended up doing if they hadn’t existed.

Agency in the free market

I’m writing this blog post at 1AM right now. In order to do so I’ve had to block twitter, close messenger, and set my phone to night mode. Luckily I’m not battling any video game or youtube shorts addictions, or this post wouldn’t exist. I don’t think I’d be able to sit down and write for hours if I had spent my entire adolescence on tiktok.

source

As consumer markets become more efficient and we become more skilled at capturing and retaining the attention of the populace, we should expect the average agency of society to decrease.

This is obvious if you think about it for a bit (the goal of almost every app on your phone is to get you to do less of anything which pattern matches ‘not using their app’), but there are few who appreciate the magnitude of this effect as the masses of society engrossed in video games and social media at home alone in their bedrooms every night are well-hidden from us. You can walk to the park to see 50 people enjoying the outdoors, but the millions currently scrolling tiktok at home are hidden from you in every way except via a statistic.

I love how easy generative AI makes it to learn – I sometimes talk to Claude until I’m exhausted and have to sleep. But in the free market, Claude doesn’t stand a chance against Tiktok. This post isn’t about Tiktok either though, as Tiktok doesn’t stand a chance against SuperTiktok (soon).

Outlier Success

Many have wondered why there’s fewer entrepreneurs in their 20s on a path to outlier success than there were in previous decades. Facebook IPO’d at a valuation of $104 billion when Mark was 28. Stripe reached a valuation of $35 billion when John was 29. Snapchat IPO’d at $24 billion when Evan was 25. These are outlier examples, but that’s the entire point. Where are the current outlier examples from the next generation?

I offer several potential answers:

  1. They are currently busy playing video games (which likely become more important to them at a very young age)
  2. They watched so much tiktok as a adolescent that they no longer have the attention span to build things (that would involve not using tiktok)
  3. They have so little free time due to attention economics that they no longer have original ideas (time spent doing ‘nothing’ is very valuable for quality long-term life outcomes)

Many in the tech ecosystem call me a doomer when I suggest these explanations – they certainly aren’t addicted to video games, and their friends certainly don’t have reduced attention spans from tiktok. But they live in a bubble within a bubble (context: I work in AI in San Francisco), and from my point of view the data points to these hypotheses as strong contenders. Sometimes I talk about this with normal people and they think the above is so obvious that it doesn’t even interest them. Perhaps our future has always been that of bread and circuses?

To clarify – I don’t intend to say that consumption or video games are bad; I love both of them myself! But we may be getting too good at consumer app optimization, and when that is paired with adoption at a young enough age, the outcome is undesirable. Building minecraft inside of minecraft is cool, but when I see a toddler scrolling youtube shorts on an ipad alone I feel really bad.

source

As the agency of the average consumer decreases, the ceiling for the agency of outliers increases. Examples of inventions which drastically increased the ceiling of agency include venture capital, generative AI, programming, microchips, and trade and capitalism itself. Once AI agents start to actually work it seems like this will be another large driving force here. Many have wondered when the first one-person billion-dollar company will exist, and many predict it may be within just one or two decades. It will be an interesting time to be alive in, if nothing else.

If you enjoyed this post, you may also enjoy: Types Of Memetic Information and my Home Page.

Types of Memetic Information

Posted on by

Three types of memetic information:

  • memetic-invariant information remains constant in value if more people learn it
  • memetic-cooperative information becomes more valuable if more people learn it
  • memetic-competitive information becomes less valuable if more people learn it

Memetic-invariant

Most simple facts are memetic-invariant. That the sky is blue or that water takes the shape of its container continue to be true and have similar value no matter how many people learn it.

Memetic-cooperative

Sometimes we are lucky enough to discover memetic-cooperative information, which becomes more valuable the more people learn it. Most of our society is built upon the fact that memetic-cooperative information exists, is easily socially transmissible, and can lead to stable equilibria built upon common knowledge.

The rule of law has value because of the social consensus that it will be enforced. When a crime is committed, it is often because someone thinks they will get away with it. If everyone stops believing that the rule of law will be enforced, anarchy may break out, and society may cease to exist.

Similarly, centralized fiat currencies are able to be used at scale and over long periods due to the mutual information between market participants that the currency has value. It is in both of our interests that we believe the dollar has value, as that allows us to easily transact with each other (See also: network effects, common knowledge, social equilibrium, minimum viable superorganism).

The general principles of how to found a company are usually memetic-cooperative: it is good for us if we make it easy to found businesses and create new things, generally speaking.

Memetic-competitive

Much of the most useful information, however, is memetic-competitive: its value decrease the more people that find it. If you have ever thought of an idea that was so good that you were reluctant to share it with anyone, that’s a sign it was memetic-competitive.

An obvious example of a memetic-competitive idea is a market undervaluing an asset. If you have an easy way of making money by finding under-priced assets, this will stop working if you share it with enough people, as eventually the inefficiency will be fully exploited. You have a true idea that may literally cease to be true if you share it with enough people.

Those knowledgeable of the basics of game theory may find that some (but not all) cases of memetic-competitive ideas map onto defection. I could discover that if I cut to the front of the line, no one will stop me, and I can selfishly save myself time. If everyone implements this strategy, the entire queuing system will break. But even if they don’t, I am taking time away from them in order to give it to myself. Systems usually solve defection like this by making games iterated: if everyone in my town knows I’m the type of person to cut in line, the reputational hit may harm me enough to serve as deterrence (See also: iterated prisoner’s dilemma, free-rider problem, the virtue of silence).

Memetic-competitive information is a subset of anti-memetic information (information which ‘does not want’ to be shared). Another example of anti-memetic information is information which has little or negative value to a majority of agents in an ecosystem (if I posted a long random string on social media, it would probably not be picked up by any meme-promulgating algorithms and is thus anti-memetic).

Execution specifics of recently successful go-to-market strategies for consumer startups are often memetic-competitive. While it’s common to share generally good sales and advertising advice, it is uncommon for the specific tactics that startups tediously searched for and executed upon to be fully divulged to potential competitors (See also: things I wish more founders would understand about b2c marketing).

Social Media

Most social media (and the broader information economy) exists to propagate memetic information. As memetic-competitive information is anti-memetic (no one wants to spread it, lest it loses value), it follows that it is not something which will be easy to find on social media (See also: alpha used both in its original financial context, but also in non-financial contexts of finding strategies which give you an edge over competitors).

For an uncommon enough individual, it may be challenging to find the most useful information, because you first have to find a community of other individuals that value the same thing. Furthermore, if the information you are after is memetic-competitive, this can only exist in small communities, so you will have to hunt them out. Unfortunately it follows that it will be hard to hunt them out, because if the community has no barriers to entry and lets anyone join, then all the memetic-competitive information they share will quickly lose its value and the community will cease to exist. Luckily not all information is of the memetic-competitive type, although there’s still many other dynamics of virality to consider (See also: geeks, mops, and sociopaths in subculture evolution on the dynamics of community change over time as new actors with different incentives join, the melancholy of subculture society on internet subculture, and border stories: why is it that every unit of life from the cell to the immune system to the country to the planet all seem to have borders).

Further Complexity

Some information can move between different memetic information categories depending on its truth value. For example, take the sentence “AI is going to kill us”. If this sentence is true, then it is memetic-cooperative: we want everyone to know about it so that we can work together to stop it. If this sentence is false, however, it could be (partially) memetic-competitive: perhaps I want you to believe AI is dangerous so that I am able to profit from AI while you are not (I don’t think this is currently the case for most actors).

Things get complicated when we introduce the fact that not only do different agents have different values, but the perceived truth value of statements will also differ between agents (and especially) groups. If we anted to we could also construct a model of memetics which is separate from the delta of informational value that propagation of the meme causes, if we wanted to get closer to fully modeling modern information exchange (See basic reproduction number R0, which is in my opinion the best way to model internet memes).

I’m going to stop the post here before this attempted formalism is over-extended, as my primary motivation is only to publicize the three labels I choose for future reference. My hope is that this post is memetic-cooperative!

See also: home page

LLMs are strangely-shaped tools

Posted on by

few hit AI apps exist because LLMs are what i’d call very “strangely-shaped” tools.

most tools are built with a specific purpose in mind like a screwdriver or a car.

but LLMs were something we stumbled upon by predicting text and playing with RL – we didn’t design the shape of them beforehand, we just let them naturally evolve into what the loss optimized for.

it’s clear they’re good at many things, but they are so strangely shaped that it’s easy to fall into traps when making products.

AI agents are a good example of a trap (for now), where it’s easy to spend months trying to perfect your scaffolding yet never quite reaching the level of reliability you’d hope for.

long-term memory implemented solely via RAG is another trap. it’s just tempting enough to try, but the results aren’t as good as they should be.

other common inadequacies include poor search, hallucinations, and high inference costs. but there’s a long list of subtle weaknesses which few tinkerers ever notice as well as many weaknesses (and strengths) which remain unfound.

much of the frontier of LLM posttraining is currently concerned with these inadequacies – wondering how we can mold these strangely-shaped LLMs we have grown into a slightly more suitable form for the problems we face.

this is hard, even for the major labs. as we slowly progress on it, i’d expect to continue to see most AI products attempt to solve the same problems via the same methods, further suffering from lack of distinctness both in performance and aesthetic, because they don’t have the right connection between research teams and product teams (or perhaps the right vision to begin with).

it’s telling that among the few recent consumer successes like midjourney or perplexity, competitors are hyper-focused on directly copying winners rather than exploring the vast new frontier of things which could be built instead. this makes sense because the frontier is strangely-shaped, as a result of the underlying catalyst itself being strangely-shaped.

it’s not uncommon for services to launch a feature literally called “AI” which is primarily composed of literal magic wands and glitter emoji simply because the product designer has no idea how to actually convey the intended experience to the user. 2024 is certainly not a year one would be fired for using too much AI.

I expect it to get more interesting later this year and especially in 2025, but it’s still been a surreal experience continually contrasting my day to day life in san francisco with that of the actual real world (note: SF is not real in this example).

the above is also relevant to some of the reasons i have longer agi timelines than i did a few years ago. agi is not a strangely-shaped tool. in fact, it is quite literally the opposite.

Originally posted via Twitter/X, but mirrored here for convenience.

If you liked this post you may be interested in the rest of my website too!

Quotes from The Everything Store – Jeff Bezos and the Age of Amazon

Posted on by

The Everything Store – Jeff Bezos and the Age of Amazon by Brad Stone is a book detailing some factors that led to the rise of Amazon as one of the largest corporate success stories of all time. I opened it expecting to skim through some parts, but ended up reading it in full in one sitting, and enjoyed it thoroughly. It left me with a strong sense of what makes Amazon, well, Amazon. And the best answer to that question is without a doubt, Bezos himself.

Rather than a full book review, I’m going to share some quotes from The Everything Store that stood out to me. One fun thing to note is that when this book was published in 2013, Amazon was ‘only’ a $150B company, but today is worth over 1.5 trillion. It’s a wonderful book and is worth buying if you want to read stories about Jeff Bezos’ extreme confidence in himself and his company as they overcome challenges one after another full-speed ahead, from local stores to Barnes and Noble to Ebay to Walmart and beyond. Below are some quotes that I particularly liked, only from the first fourth of the book. I was quoting the book more than anticipated, so I stopped this post early but will leave it up as an advertisement for the book.

Bezos is an excruciatingly prudent communicator for his own company.
He is sphinxlike with details of his plans, keeping thoughts and intentions
private, and he’s an enigma in the Seattle business community and in the
broader technology industry. He rarely speaks at conferences and gives
media interviews infrequently.

There is so much stuff that has yet to be invented.
There’s so much new that’s going to happen.
People don’t have any idea yet how impactful the Internet is going to
be and that this is still Day 1 in such a big way.
Jeff Bezos

Amazon’s internal customs are deeply idiosyncratic. PowerPoint decks
or slide presentations are never used in meetings. Instead, employees are
required to write six-page narratives laying out their points in prose, because
Bezos believes doing so fosters critical thinking. For each new product, they
craft their documents in the style of a press release. The goal is to frame a
proposed initiative in the way a customer might hear about it for the first
time. Each meeting begins with everyone silently reading the document, and
discussion commences afterward

“If you want to get to the truth about what makes us different, it’s this,”
Bezos says, veering into a familiar Jeffism: “We are genuinely customer-
centric, we are genuinely long-term oriented and we genuinely like to invent.
Most companies are not those things. They are focused on the competitor,
rather than the customer. They want to work on things that will pay
dividends in two or three years, and if they don’t work in two or three years
they will move on to something else. And they prefer to be close-followers
rather than inventors, because it’s safer. So if you want to capture the truth
about Amazon, that is why we are different. Very few companies have all of
those three elements.”

Bezos interpolated from this that Web activity overall had gone up that year by a factor of roughly 2,300—a 230,000 percent increase. “Things just don’t grow that fast,” Bezos later said. “It’s highly unusual, and that started me thinking, What kind of business plan might make sense in the context of that growth?”

Jackie Bezos suggested to her son that he run his new company at night or on the weekends. “No, things are changing fast,” Bezos told her. “I need to move quickly.”

Internet records show that during that time, they registered the Web domains Awake.com, Browse.com, and Bookmall.com. Bezos also briefly considered Aard.com, from a Dutch word, as a way to stake a claim at the top of most listings of websites, which at the time were arranged alphabetically.

Bezos and his wife grew fond of another possibility: Relentless.com. Friends suggested that it sounded a bit sinister. But something about it must have captivated Bezos: he registered the URL in September 1994, and he kept it. Type Relentless.com into the Web today and it takes you to Amazon.

They set up shop in the converted garage of Bezos’s house, an enclosed space without insulation and with a large, black potbellied stove at its center. Bezos built the first two desks out of sixty-dollar blond-wood doors from Home Depot, an endeavor that later carried almost biblical significance at Amazon, like Noah building the ark.

During that time, the name Cadabra lived on, serving as a temporary placeholder. But in late October of 1994, Bezos pored through the A section of the dictionary and had an epiphany when he reached the word Amazon. Earth’s largest river; Earth’s largest bookstore.3 He walked into the garage one morning and informed his colleagues of the company’s new name. He gave the impression that he didn’t care to hear anyone’s opinion on it, and he registered the new URL on November 1, 1994. “This is not only the largest river in the world, it’s many times larger than the next biggest river. It blows all other rivers away,” Bezos said.

One early challenge was that the book distributors required retailers to order ten books at a time. Amazon didn’t yet have that kind of sales volume, and Bezos later enjoyed telling the story of how he got around it. “We found a loophole,” he said. “Their systems were programmed in such a way that you didn’t have to receive ten books, you only had to order ten books. So we found an obscure book about lichens that they had in their system but was out of stock. We began ordering the one book we wanted and nine copies of the lichen book. They would ship out the book we needed and a note that said, ‘Sorry, but we’re out of the lichen book.’

A week after the launch, Jerry Yang and David Filo, Stanford graduate students, wrote them an e-mail and asked if they would like to be featured on a site called Yahoo that listed cool things on the Web. At that time, Yahoo was one of the most highly trafficked sites on the Web and the default home page for many of the Internet’s earliest users.

In the meetings, Bezos presented what was, at best, an ambiguous picture of Amazon’s future. At the time, it had about $139,000 in assets, $69,000 of which was in cash. The company had lost $52,000 in 1994 and was on track to lose another $300,000 that year. Against that meager start, Bezos would tell investors he projected $74 million in sales by 2000 if things went moderately well, and $114 million in sales if they went much better than expected. (Actual net sales in 2000: $1.64 billion.)

Bezos later told the online journal of the Wharton School, “We got the normal comments from well-meaning people who basically didn’t believe the business plan; they just didn’t think it would work.”11 Among the concerns was this prediction: “If you’re successful, you’re going to need a warehouse the size of the Library of Congress,” one investor told him.

When his goals did slip out, they were improbably grandiose. Though the startup’s focus was clearly on books, Davis recalls Bezos saying he wanted to build “the next Sears,” a lasting company that was a major force in retail. Lovejoy, a kayaking enthusiast, remembers Bezos telling him that he envisioned a day when the site would sell not only books about kayaks but kayaks themselves, subscriptions to kayaking magazines, and reservations for kayaking trips—everything related to the sport. “I thought he was a little bit crazy,” says Lovejoy.

The IPO process was painful in another way: During the seven-week SEC-mandated “quiet period,” Bezos was not permitted to talk to the press. “I can’t believe we have to delay our business by seven years,” he complained, equating weeks to years because he believed that the Internet was evolving at such an accelerated rate. Staying out of the press soon became even more difficult. Three days before Amazon’s IPO, Barnes & Noble filed a lawsuit against Amazon in federal court alleging that Amazon was falsely advertising itself to be the Earth’s Largest Bookstore. Riggio was appropriately worried about Amazon, but with the lawsuit he ended up giving his smaller competitor more attention. Later that month, the Riggios unveiled their own website, and many seemed ready to see Amazon crushed. The CEO of Forrester Research, a widely followed technology research firm, issued a report in which he called the company “Amazon.Toast.”


It was a distilled version of the dissatisfaction felt by many early Amazon employees. With his convincing gospel, Bezos had persuaded them all to have faith, and they were richly rewarded as a result. Then the steely-eyed founder replaced them with a new and more experienced group of believers. Watching the company move on without them gave these employees a gnawing sensation, as if their child had left home and moved in with another family. But in the end, as Bezos made abundantly clear to Shel Kaphan,family. But in the end, as Bezos made abundantly clear to Shel Kaphan, Amazon had only one true parent.

“You seem like a really nice guy, so don’t take this the wrong way, but you really need to sell to Barnes and Noble and get out now,” one student bluntly informed Bezos. Brian Birtwistle, a student in the class, recalls that Bezos was humble and circumspect. “You may be right,” Amazon’s founder told the students. “But I think you might be underestimating the degree to which established brick-and-mortar business, or any company that might be used to doing things a certain way, will find it hard to be nimble or to focus attention on a new channel. I guess we’ll see.”

“There will be a proliferation of companies in this space and most will die. There will be only a few enduring brands, and we will be one of them.”

During that time, no one placed bigger, bolder bets on the Internet than Jeff Bezos. Bezos believed more than anyone that the Web would change the landscape for companies and customers, so he sprinted ahead without the least hesitation. “I think our company is undervalued” became another oft- repeated Jeffism. “The world just doesn’t understand what Amazon is going to be.”

As the company grew, Bezos offered another sign that his ambitions were larger than anyone had suspected. He started hiring more Walmart executives.

Around that time, Wright showed Bezos the blueprints for a new warehouse in Fernley, Nevada, thirty miles east of Reno. The founder’s eyes lit up. “This is beautiful, Jimmy,” Bezos said. Wright asked who he needed to show the plans to and what kind of return on investment he would have to demonstrate. “Don’t worry about that,” Bezos said. “Just get it built.” “Don’t I have to get approval to do this?” Wright asked. “You just did,” Bezos said. Over the next year, Wright went on a wild $300 million spending spree.

“Walmart did not even have Internet in the building back then,” says Kerry Morris, a product buyer who moved from Walmart to Amazon. “We weren’t online. We weren’t e-mailing. None of us even knew what he meant by online retail.”

The venture capitalists backing eBay asked around and heard that one did not work with Jeff Bezos; one worked for him.

Bezos went skiing in Aspen that winter with Cook and Doerr and finally told them what was coming. “He said, ‘We’re going to win, so you probably want to consider whether to stay on the eBay board,’ ” says Cook. “He thought it would be the only natural outcome.”

If you liked these quotes, consider reading the full copy (perhaps even buying it from Amazon), it’s definitely a nice read about an amazing company and individual.