M&A, Before and After: What Founders Need to Know

Martin Casado, Blake Kim, and Zoran Basich

Welcome to the a16z podcast. Today we’re talking about the mindsets and frameworks founders should know about when navigating the mergers and acquisitions or M&A process, both before and after – including how to think about the pricing dynamics, factors that go into the decision-making process, and what to expect from the integration once the deal is done.

A16z editorial partner Zoran Basich recently talked to two a16z experts here to give us their big-picture view of the most important things to know – for founders seeking to acquire companies and how they might think about it, or those considering selling a company, or just those deciding to merge with an acquirer.

Blake Kim is a partner on our Enterprise Network team and a former investment banker who works with companies on strategic partnerships; he also recently co-wrote a post on Future outlining all the different exit options and considerations for companies. And general partner Martin Casado discusses common M&A issues and shares his experiences both as observer and participant – including the challenges of integration, which he saw from the inside with Nicira, which he cofounded and was acquired by VMware for $1.26 billion in 2012, and where he remained for years to lead its networking and security business unit.

As a reminder, none of the following should be taken as investment advice. Please see a16z.com/disclosures for more important information.

They start the discussion by outlining the frameworks for understanding M&A dynamics, including the “kingmaking dynamic” and the difference between “selling your company” and “getting acquired.”

  • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

  • Blake Kim is Enterprise Lead in a16z's Capital Network Group, advising companies on capital raising, M&A, and public markets. He has worked at Bear Stearns, Lehman Brothers and Thomas Weisel Partners/Stifel.

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

Crypto at Congress: ‘Watershed’ Moment for Regulation and Web3

Tomicah Tillemann and Zoran Basich

Welcome to 16 Minutes, our podcast where we discuss tech trends in the news and their impact on the long arc of innovation. Today’s topic is crypto regulation, and specifically, two recent federal government hearings in the news that were focused on crypto and therefore the related trend of web3. In contrast to the model of web2 — typified by very broadly used but also very centralized platforms run by corporations — web3 refers to the idea of a new internet enabled by crypto that is owned by builders and users.

The first hearing that took place was at the House Committee on Financial Services, featuring six crypto company CEOs and resulting in a five-hour session that prompted headlines like “Congress Gets a Crash Course on Cryptocurrency.”

Then, the U.S. Senate’s Banking, Housing, and Urban Affairs Committee held its own hearing, this time focused on stablecoins, which are privately issued cryptocurrencies that are pegged to a stable asset such as the U.S. dollar, and are used in decentralized financial services.

We’ve covered crypto regulatory issues on 16 Minutes before with a16z experts, including an episode with former federal prosecutor  Katie Haun and former New York Stock Exchange regulatory chief Anthony Albanese. That discussion, which you can find in this feed under episode #50, was about a proposal by the Treasury Department’s financial crimes enforcement arm that included provisions for digital asset reporting.

(As a reminder, none of the following should be taken as investment advice, please see a16z.com/disclosures for more important information.)

All of these hearings are also connected to the broader question of innovation, and keeping the U.S. competitive on a global stage.

So with that context, our guest today is a16z global head of policy Tomicah Tillemann, who before joining a16z served as senior advisor to two secretaries of state. He reports on the hearings and their significance, and gives a quick pulse-check on where we are with crypto regulation right now.

  • Tomicah Tillemann is global head of policy at a16z. Previously he served as a senior advisor to now-President Joseph Biden and two secretaries of state.

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

How ‘Hyperscalers’ are Innovating — and Competing — in the Data Center

Nick McKeown, Martin Casado, and Zoran Basich

Innovation in the data center has been constrained by the traditional model of suppliers providing fixed-function chips that limit how much the biggest data center operators can differentiate. But programmable chips have emerged that allow these companies to not only increase performance, but innovate throughout the pipeline, from operating system to networking interface to user application.

This is a major trend among “hyperscalers,” which are some of the world’s most well known companies running massive data centers with tens of thousands of servers. We’re talking about companies like Amazon, Facebook, Microsoft, Google, Apple, Alibaba, Tencent.

To talk about the trends in data centers and how software may be “eating the world of the data center,” we talked this summer to two experts. Martin Casado is an a16z general partner focused on enterprise investing. Before that he was a pioneer in the software-defined networking movement and the cofounder of Nicira, which was acquired by VMWare. (Martin has written frequently on infrastructure and data-center issues and has appeared on many a16z podcasts on these topics.)

He’s joined by Nick McKeown, a Stanford professor of computer science who has founded multiple companies (and was Martin’s cofounder at Nicira) and has worked with hyperscalers to innovate within their data centers. After this podcast was recorded, Nick was appointed Senior Vice President and General Manager of a new Intel organization, the Network and Edge Group. The podcast begins with Nick, talking about the sheer scale of data-center traffic. 

  • Nick McKeown

  • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

Kickstarting Network Effects

Paul Davison, Alexis Ohanian, Andrew Chen, and Das Rush

Network effects can be found powering almost every major technology company, from messaging apps and workplace collaboration tools, like Slack and Zoom, to marketplaces, like Airbnb and Instacart to even the internet itself. In this podcast, we look at the role of network effects creator-driven social platforms, with Alexis Ohanian, cofounder from Reddit, Paul Davison, cofounder from Clubhouse, and a16z general partner Andrew Chen, whose new book, “The Cold Start Problem: How to Start and Scale Network Effects” comes out this week (see coldstart.com for more). We cover: how do you cold start and get your first creators? How does your relationship to creators change as you scale? And how is web3 changing the incentives and dynamics around network effects?


  • What are network effects? [1:32]
  • How do you cold start and get your first users? [2:33]
  • Atomic networks and why minimum viable community is more important than minimum viable product [6:36]
  • How do you curate your network and set norms? [8:42]
  • Faking users: good idea, bad idea? [13:13]
  • What is flintstoning? [14:26]
  • How does the relationship to creators change as you scale? [17:07]
  • Building for the professional creator class [22:52]
  • How is web3 changing incentives? [25:12]
  • Paul Davison Paul is the cofounder of Clubhouse

  • Alexis Ohanian Alexis was a cofounder of Reddit and is now a business dad and an investor and mentor to entrepreneurs.

  • Andrew Chen is a GP at a16z in consumer technology. Prior to that, he led Rider Growth at Uber. He has written for a decade on metrics & growth on his blog/newsletter, and is the author of The Cold Start Problem.

  • Das Rush

Inside the GameStop Drama; U.S. Constitution, Auctioned

Ken Griffin and Marc Andreessen

Welcome to 16 Minutes, our show on the a16z podcast network where we talk about tech trends that are dominating news headlines, industry buzz, and where we are on the long arc of innovation. 

Today’s episode actually features a look back at the GameStop saga — the stock market drama that some headlines described as a “David-and-Goliath battle” that “upended Wall Street.” 

For quick basic context, here’s what happened: A group of Reddit users mass-purchased and drove up prices of stock in the video game retailer GameStop, forcing short sellers including hedge funds and institutional investors to back out in a short squeeze, pushing prices even higher. But beyond the news, this also portended other, broader trends including redefining the power of retail investors, the phenomenon of meme stocks, and more.  

So in this episode — which is from a conversation that originally took place live on Clubhouse  (and which, by the way, can also be found on the a16z Live feed) —  a16z co-founder Marc Andreessen talks to Ken Griffin, founder and CEO of the hedge fund Citadel, which was a key player in GameStop as both a market maker and investor. You’ll also hear a16z general partner and fintech expert Alex Rampell join later in the conversation.

Griffin also just purchased (in a Sotheby’s auction a little over two weeks ago) — one of the original copies of the U.S. Constitution, an auction in which a decentralized autonomous organization called ConstitutionDAO also bid on buying it; Marc and Ken discuss this briefly at the very end.

  • Ken Griffin

  • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

‘Play-to-Earn’ Gaming and How Work is Evolving in Web3

Arianna Simpson, Gabby Dizon, Jeffrey Zirlin, and Zoran Basich

In today’s episode we’re talking about an emerging model of gaming called play to earn, in which players can make actual money based on how much time and effort they put into a game. Play to earn is also part of broader trends — the changing relationship between players and platforms, new incentives for participants in blockchain-based networks, and the new internet era that is coming to be known as a web3.

The top play-to-earn game is called Axie Infinity, operated by a Vietnam-based company called Sky Mavis. Players of the game acquire unique digital pets called Axies, and battle other teams of Axies. These NFT Axies can be created and sold using the game’s in-game currency, SLP, which can be traded for traditional currency.

Think of it as Pokemon on the blockchain, with a social network built-in, and an actual economy, and even companies built around the game that help players onboard and loan them money to get started playing. The game has made more than $3 billion in total sales since launching in March 2018, with much of its early growth in the Philippines.

(As a reminder, none of the following should be taken as investment advice, please see a16z.com/disclosures for more important information.)

Our guests today are Jeff Zirlin, the cofounder of Sky Mavis; Gabby Dizon, the cofounder of Yield Guild Games, a play-to-earn-gaming guild that gives players the resources to start playing; and a16z crypto general partner Arianna Simpson.

They talk to a16z’s Zoran Basich about the larger tech trends that enabled the emergence of play to earn, why and where it caught on first, and the role of community, as well as the challenges, which include onboarding and scalability, and the economic sustainability of this model.

Transcript below:

Axie and the beginnings of play-to-earn

Jeff: This is something that took almost four years to marinate. So, a lot of the original Axie community members and founders were from the CryptoKitty community, where we saw that the production of new NFTs had to be related in some ways to actual work and effort. This would basically prevent hyperinflation. So in the first iteration of the Axie battle system, we had experience points and these experience points could be used to breed Axies. We then had the idea and a lot of our players were requesting that these experience points be tradable tokens. And so, we tokenized them.

And then we saw that our players had actually created a liquidity pool on Uniswap, which is a decentralized exchange for these tokens. We then realized that, hey, you can actually calculate an expected hourly wage for playing Axie because there’s guaranteed liquidity in this pool.

So, I think that was really the moment that we saw the potential for it. We put out a post in January of 2020 where we saw that there was this intersection of NFTs and DeFi that was creating something that we then dubbed play-to-earn, and I think that might be the first time that that terminology was used, at least the first time where it was used in a non-speculative way

Zoran: One key thing to note here is that people have to pay a significant amount of money to get started – they have to download crypto wallets and buy three Axies, which can cost upwards of $1,000, with prices fluctuating. So Gabby, you saw this from the ground in the Philiipines — what did you see in this economic model?

Gabby: So I’ve been in the game industry for almost 20 years and actually joined the Axie community in late 2018 as a player. And what I saw last year is that people from my home country in the Philippines had started discovering and playing Axie as a way to kind of escape the economic hardship of the lockdown. And these were people who weren’t crypto enthusiasts. These were regular people that were stuck at home that had no jobs.

So, came upon the idea of a guild as a way to scale the efforts to introduce people to the world of play-to-earn and provide Axies via a scholarship program that would provide access to people around the world, not just from the Philippines, but across Southeast Asia, in India, in Latin America, and other countries to be able to play the game and earn money from it without having to afford the assets upfront.

And what we do is that we kind of act as a player collective where we invest in a lot of these assets and then form the communities around these different games where people can participate into these networks so that it’s not anymore a hurdle to get into a game like Axie Infinity because I can’t afford three Axies.

The conditions for growth — the Philippines and beyond

Zoran: You touched on the growth in places like the Philippines. What were the conditions that enabled that? Why did that spark catch there? And what does that tell us about building communities in the future?

Jeff: I think that there are a couple of unique things related to the Philippines. The Philippines loves mobile games. There’s relatively high crypto literacy. It’s a very communal culture where information and trends can spread very quickly. Filipinos have traditionally been early adopters to many social networks and platforms like Facebook. With the Philippines specifically, I think there were a lot of really special circumstances that basically allowed it to break out ahead of the rest of the world, but I do think that it’s only around nine months ahead of the rest of the emerging market economies

How play-to-earn reflects broader web3 trends

Zoran: So if this model indeed spreads and continues to grow, that suggests there’s something bigger going on here – what is that, what transition are we actually seeing?

Arianna: We think that the people creating the value should be participating in the upside. And that’s really the core belief at the center of what’s happening with web3. People are earning their livelihood on Axie, and so having the ability to not have all the value retained by the platform, but actually, pass it back to the community, the creators, the people who, again, are really responsible for building the value is, I think, an unstoppable movement

Jeff: One of the frameworks that we can think about is, like, okay, who are the middlemen that are getting removed? And how is the value that was being extracted by those middlemen shared with the actual users and stakeholders of the platform? So with Axie, I think what’s happened is that the app stores and the game publishers have been removed from the equation. They traditionally take 50%-plus of the revenue that’s generated by a game. We’ve removed them. We don’t use them. And we’re sharing that value with the people who are actually driving traction for our network, which is the community.

How sustainable is it?

Zoran: Okay, people are paying to get in and they are interacting with their characters, and they’re competing, and they’re earning rewards. But help me understand the sustainability of that. How do you think about the underlying economic model here?

Jeff: So one of the ways to look at it is right now, Axie is a little bit of a growth-dependent economy, just like any emerging market nation. It is a little bit dependent on capital inflows. But long term, it’s really important for us to have players that are in the economy spending because they think that the game is really fun or that they see ways to trade, like, money for power or respect. And the more Axie becomes like a real social network, a nation, the more opportunities for those types of value exchanges to arise. And we have framed Axie as a social network as well as a game since the early days. And this is becoming more and more true as, you know, we’ve done things like max out our Discord.

Zoran: What are the key factors to making play-to-earn sustainable? Like, how do you think about potential ceilings on capital inflows and participation?

Arianna: In many ways, these economies are going to be similar to real-world economies. They’re very complex and I think continuing to manage the capital and token supply and all of the dynamics within these metaverses is going to be a real challenge.

One of the key things in my mind is making sure that there continue to be different groups of players who are participating for different reasons and deriving different kinds of value. So, people might be there purely because they enjoy playing the game or because they get a lot of personal fulfillment out of coaching others. In many cases, it might be financial. There are a number of different reasons why people might want to be playing the game and engaging in the community. But as long as the players who are putting in additional capital derive other kinds of enjoyment from it, that’s fine. And that’s actually not at all dissimilar to the real-world economy. I might be wealthier and I might pay someone to do something I don’t want to do and somebody might, in exchange, pay me to do something they don’t want to do.

And so as long as there continues to be a robust ecosystem of participants who are there for a variety of reasons, I think that really drives things forward.

Zoran: What is the balance between people who are trading within the economy versus taking money out of the economy, like, you know, transferring it to fiat, because people are paying their bills, right? And so they have a decision to make about how much to kind of put back into the system versus take out. What are the trends there? And how does that affect the business model?

Jeff: In terms of the capital flows, we see that there are far more funds that are being deposited into our ecosystems than withdrawn. Only 4% of Axies are for sale on the marketplace. So, I think this shows the emotional connection and the fact that people don’t just see this about money. It’s also something that is giving them access to new opportunities, social networks, and is a lot of fun

So, right now, I think, it’s easier to balance the economy when we’re growing really quickly and.the animal spirits are active. Long term it will be about continuing to make sure that the community is insanely fun to be a part of and the game is insanely fun to play.

The role of community

Zoran: But what is it that keep people feeling as though they’re part of a community? There is some bigger, evangelizing feeling here than just, “Yeah, I’m gonna make a few bucks.” Right?

Jeff: Sure. I think it boils down to this shared economic alignment, but also cultural alignment. This community has been around since before NFTs were popular and, you know, what everyone in the world was interested in. Right? So, this exploration, I think that’s part of the DNA of the community. And as new entrants come in, they learn about this and they actually have some of the ideals transferred into them as they join. So, I think that’s been really important. And the community is amazing at the education and the onboarding. Eighty percent of our users are coming from referrals.

Arianna: I think it just demonstrates the enthusiasm that people have for what’s happening, both in physical space, by the way — the meetups have incredible attendance, people are really organizing themselves on the ground — but also, of course, in the digital realm, so in Axie itself, in the Discord, in the Substacks, etc. So, there’s just a stickiness that comes from people feeling like this community is theirs and they are benefiting from being members of it rather than, you know, having someone extracting value from them.

Zoran: But it seems like there’s kind of a paradox here, right? Because one of the things about legacy systems, whether it’s financial systems or gaming, is that they’re very sticky. It’s hard to move your account from one bank to another or you can’t move your in-game goods from one game to another. And crypto has kind of changed that and it’s made it much more portable. Isn’t it easy for some other game to pop up that is just as appealing and the characters are just as cute and your community might migrate over there?

Jeff: What we’ve built is not just the gaming community, it is, in many ways, a nation where people have shared cultural values, there’s overlaps and entertainment, there’s even this lingo or jargon similar to language. We have this very deep economy. So, I think it is much harder with this type of network to uproot a community and transplant them into a new universe where they don’t have a stake in it. We’re seeing people in the real world form Axie communities where you know all the people in your town or your city who plays Axie Infinity. And it has network effects, right? The more people that own Axie tokens, the more people that own Axies, the more people that own land within the universe, the deeper entrenched these economic and social relationships get with each other.

Gabby: Yeah. One of the hallmarks of the shift between web2 to web3 is that the communities are opt-in and they’re incentive-aligned by shared economic ownership, by the kind of traits that lead people to share the same affiliation, the same tribe around maybe certain assets or certain game universe. People stay here because they choose to be here, and they help build the culture of a tribe. There’s a shared economic incentive, there’s a cultural incentive. But if people want to quit that network and leave and join another network, no one is preventing them to do so.

Zoran: So speaking of user choices and user behavior, how is that changing? Because YGG is funding players, bringing them into the game, helping them get started, you’ve got a close-up view of this – is there a pattern of behavior, or different modes of participation that are evolving?

Gabby: We see ourselves as a necessary layer to bring people from the real world into the metaverse, especially for those that can’t afford it. But once they’re there and have an income, we actually encourage them to turn people from gamers to investors.

So, the first few cycles of people are earning money via SLP. They sell it for fiat, put food on the table, pay their bills. What we see is that when they’re a few cycles in and they have some excess income, many of them for the first time in their lives, the behavioral patterns actually change. People have more of an investor mindset. And now they think about, “Do I buy Axies for myself and graduate from this program so that the next person can benefit from the Axies that I was using? Do I buy Axies for my family members so that they can start earning money too? Do I buy my first piece of land in Axie Infinity and become a virtual landowner?” And so on and so forth.

Scalability challenges and Ronin

Zoran: Jeff, you mentioned CryptoKitties earlier. That’s the most prominent example before this of a game that really reached some level of mainstream success. And it kind of brought down Ethereum for a while, right? So in terms of growth and scalability, what are the challenges?

Jeff: In our ecosystem we have a lot of small, kind of low-value transactions that are really key to our user base in the emerging markets. And these were things that were basically priced out on Ethereum, where on Ethereum they might be $5 to $15 and obviously we’ve been in a bull market and that’s also coincided with rising gas costs on Ethereum as well. We were at the mercy where the success of the Ethereum ecosystem and all these DeFi applications was actually strangling out our growth, right? It was pricing our users out. We were in a situation where we really needed to migrate the majority of our transactions onto our own infrastructure.

It’s really this April with the launch of Ronin, which is our Ethereum sidechain, where we added that key piece to the equation, which allowed for our growth. We were around 38,000 daily active users in April before the launch of Ronin. And we just hit 2.4 million daily active users.

Zoran: What exactly did Ronin, this sidechain, do for you — what did it unlock?

Jeff: It allowed for the proliferation of the scholarship model, because there are a lot of transactions that are involved in running a scholarship program. So, breeding Axies, sending Axies to different accounts so that they can be distributed to scholars, claiming in-game rewards, these are all transactions on a blockchain that on Ethereum would cost a lot and, oftentimes, take a while, whereas on Ronin, these have all become very cheap, free up until now, and much faster.

That’s why we call Ronin Ronin. Right? Ronin is the samurai without a master. This was all about taking our destiny into our own hands where we could be the ones that determined our growth path rather than having to be a taker of market conditions and gas prices on Ethereum.

Arianna: I think that just really tells a critical story. If they hadn’t done that, I think the game would have obviously continued to grow, but been capped in the way that it was before. And so once there was that infrastructure put in place, there was an incredible unlock, which is really visible from the chart.

Onboarding into the game

Zoran: And so the other challenge too is onboarding, in terms of the steps required to start playing the game. You have to download multiple wallets and you have to, obviously, pay some money. And on the one hand, that shows how appealing the game is for people that they are willing to take these steps. But on the other hand, you want it to become easier.

Jeff: It is still incredibly difficult to get started with Axie. So, a lot of our development roadmap is aimed at reducing these barriers, specifically, for example, right now, if you are interested in Axie and you want to play, you then have to figure out which Axies you actually want to buy. This might involve doing a lot of research on the internet, maybe watching YouTube tutorials. And you’re basically buying characters for a game that you’ve never played before.

We will be releasing an upgraded battle system. And one of the features of that will be kind of this demo tutorial where everyone will be able to download Axie, get a free team of starter Axies, learn about the game, figure out if they actually love the gameplay, love the community before they have to make any economic decisions. So, we think that that’s going to be a really important stage between awareness and activation.

There are also payment on-ramp and off-ramp frictions that we can work with partners on to also improve that onboarding experience.

Gabby: And, you know, That’s where the community steps in. And what I love about it is that you’re replacing the middlemen such as Facebook and Google with community-based structures that onboard people into games like Axie, teach them how to play the game, how to earn money, how to use crypto, onboard the wallet. And for me, it’s like web2 reduced people into statistics that it’s just about daily active users. And with web3, with this community-based acquisition growth returning them back into individuals, again, individuals who are like creating content, learning how to play, people who are earning money, and we see all of these stories of people whose lives were profoundly changed by earning money And I think that’s really significant.

Play-to-earn and the future of work

Zoran: The idea of work is really important here. I mean, so far, it’s players, and people funding those players. But when you squint your eyes a little bit into the future, what kinds of new jobs do you foresee, whether it’s specific jobs or categories of jobs?

Jeff: There are many different archetypes of Axie players, right? There are the competitive battlers. There are the collectors. There are the scholarship managers, as well as the scholars who are, you know, primarily kind of farming tokens and selling them within the ecosystem. There are just people who are Axie holders, there are the content creators and the educators. And many players see themselves as more than one.

I think what we’ll see, within at least the Axie ecosystem, is that the different types of gamers will start to correlate or map to different professions, right? So, there might be someone who specializes in, you know, creating consumables, like a potion maker. Might be like, I don’t know, the version of a pharmacist. You’ll have your gladiator which might be, like, similar to, your athlete. So, we’re starting to see the rise of the competitive Axie players who are able to live off of, you know, winning tournaments and climbing the leaderboard.

I think that we’ll also see politicians arise in the Axie universe — people who are leading committees, for example, and thinking about the best use of funds, that might be the treasury, that might be the ecosystem, that might be, like, putting forth governance proposals. It might be creating requests for funding for different initiatives. You might have people who are focused on accumulating or harvesting certain materials. And that might be like the version of a farmhand. Right? So, someone who’s going on to Axie land, for example, and harvesting resources. I think that’s, like, an archetype that we’ve seen in the past, but I think has never really truly broken out and I think it’s had its breakout moment with Axie and the rise of YGG and similar institutions.

Zoran: We’re hearing a lot about web3 and the metaverse, and other buzzy phrases, all kind of revolving around this idea of more and more of our lives being lived online, in increasingly deep ways. Some of these trends we’re talking about with play-to-earn, do they go beyond games, or even jobs, and just become part of how we live?

Gabby: Yeah. This intersection between crypto and the creator economy, it’s actually one of the things that excites me the most about the future of work and where people are going in the metaverse. So, when you see these games in virtual worlds, I think it will not be just in games like what we’re seeing now. There will be virtual worlds where people can be, for example, going to the bank, interacting with a DeFi application, and doing what we just think of as work but they’re doing this in these virtual worlds that may look like games, but they’re actually just the interface for doing this type of work in the virtual world. So, I think more and more the blending of work and play will come together and we’ll be using crypto as a means to exchange value, but they may not necessarily be games as we know them today.

Gaming as a pathway to crypto adoption

Zoran: And I wonder what you think about crypto adoption overall —there’s still a fair amount of skepticism by a lot of people. What does play-to-earn and gaming, in general, mean for crypto adoption?

Arianna: Gaming is going to be a key way in which the next hundreds of millions of users onboard into crypto because, historically, you know, it hasn’t been easy to get involved and the barrier to entry is pretty high, the technical hurdles are fairly high. And it can be a little bit scary. There’s real money at stake. And what games offer is a much friendlier, more approachable onboard. You have these cute digital pets. You can go on and meet other players. We’ve talked about a real sense of community. And so it’s just a much more approachable way to start playing around in the space. And so we see that players might come in through a game and then, you know, once they have a wallet, start to experiment with other web3 products and experiences, and really expand outward from there.

Zoran: And how much of the crypto will end up being under the hood in the future? Will people kind of be transacting and not even thinking about it as crypto, but, you know, buying things with credit cards? Tell me how you think about that.

Jeff: What we need to do as product developers is to make sure that our user base and our prospective users really understand what the benefits of our products are, right? The benefits are derivations of blockchain technology, I don’t think they need to understand, like, how blockchain actually creates these benefits. So, I think when they see a fun, cute game where they can actually earn real value, I think that’s good. That’s sufficient. And I think that’s a strong enough pull. I think that learning how to use blockchain, I think these are more emergent benefits that they might get. Once they get involved, you know, they might kind of fall down the rabbit hole. We love seeing our user base go through that process as well. But I think in terms of the awareness phase, the benefits should be crystal clear. Even though the onboarding is difficult, people and organizations are onboarding by hand the next generation of blockchain users.

Zoran: Arianna, Jeff, Gabby, thank you so much for being with us today.

Gabby: Thank you.

Jeff: Thanks for having us.

Arianna: Thanks, Zoran.

  • Arianna Simpson is a general partner at a16z crypto. Prior to joining, Arianna founded Autonomous Partners, an investment fund focused on cryptocurrencies and digital assets.

  • Gabby Dizon is the cofounder of Yield Guild Games, a play-to-earn gaming guild. He is an 18 year veteran of the game industry and has been in the blockchain game and NFT space since early 2018.

  • Jeffrey Zirlin is the cofounder of Sky Mavis, where he spearheads the growth of products through messaging, community empowerment, and incentive design.

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

16 Minutes: Steam Halts Web3 Games; FDA Approves Prostate Cancer AI

Jonathan Lai, Eddy Lazzarin, Vineeta Agarwala, Jay Rughani, Eliezer Van Allen, and Zoran Basich

Welcome to 16 Minutes, our show where we talk about tech trends in the news! We have two segments today:

1) The announcement recently that Valve Software, which operates the massive gaming platform Steam, added a rule barring games that use blockchain technologies or that allow users to exchange cryptocurrencies or NFTs – this rule appeared on its “What you shouldn’t publish on Steam” onboarding list for developers. We go beyond the players to the trends at play here, putting the news in context — as is the premise of this show — because it not only immediately impacts gaming developers and gamers using the platform, but has implications for gaming business models and the arc of innovation in gaming as part of the web3 movement.

Our expert guests are a16z partner Jonathan Lai and a16z partner Eddy Lazzarin.

2) The FDA’s announcement last month that it authorized marketing of the “first artificial intelligence (AI)-based software designed to identify an area of interest on the prostate biopsy image with the highest likelihood of harboring cancer so it can be reviewed further by the pathologist if the area of concern has not been identified on initial review.” The FDA reviewed the technology from Paige Prostate through its De Novo regulatory pathway.

We have three expert guests: Eli Van Allen, associate professor of Medicine at Harvard Medical School and chief of the Division of Population Sciences at the Dana-Farber Cancer Institute; a16z bio general partner Vineeta Agarwala; and a16z bio partner Jay Rughani.

  • Jonathan Lai is a partner at a16z where he focuses on games, social, and creator economy investments. Prior to joining the firm, Jon led North America investments at Tencent and was a PM at Riot Games.

  • Eddy Lazzarin is head of engineering at a16z crypto. Prior to that Eddy was a software engineer at Netflix working on data ingestion systems, and data engineer at Facebook working on growth analytics for Messenger.

  • Vineeta Agarwala MD, PhD is a general partner at a16z investing in bio and healthcare technology. She is also a practicing physician and adjunct clinical faculty member at Stanford.

  • Jay Rughani is a partner at Andreessen Horowitz investing in bio and healthcare technology companies. Prior to joining a16z bio, he worked for Flatiron Health, which was acquired by the Roche Group in 2018.

  • Eliezer Van Allen

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

Crypto Security and the New Web3 Mindsets for Users

Eddy Lazzarin and Zoran Basich

Today’s episode is all about crypto security — that is, the new mindsets and the new strategies for storing crypto assets safely while also allowing holders control and access. 

(As a reminder, none of the following should be taken as investment advice, please see a16z.com/disclosures for more important information.) 

We’ve covered security trends more broadly a ton in our content, which you can find at a16z.com/security, as well as crypto-related trends including NFTs, and the creator and ownership economies.

But as more people enter crypto lately — thanks to the boom in NFTs, decentralized finance, and much more — we share specific best practices and options for securing crypto as well as discussing how it all fits this next evolution of the internet: web3.

Our expert today is a16z crypto data scientist Eddy Lazzarin, who joins host Zoran Basich. He covers practical approaches ranging from passwords to crypto wallets and what users can do; the evolution of crypto briefly; and the big picture mindset shifts involved here as well.

  • Eddy Lazzarin is head of engineering at a16z crypto. Prior to that Eddy was a software engineer at Netflix working on data ingestion systems, and data engineer at Facebook working on growth analytics for Messenger.

  • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

Cloud Wars & Company Wars: Play Nice, But Win

Michael Dell, Marc Andreessen, Martin Casado, and Sonal Chokshi

    There are lots of challenges in being public while trying to innovate, and limits to being a private company as well; but it’s rare to see a company go public then private then back to public again. As is the case with Dell Technologies, one of the largest tech companies — which went private 2012-2013 and then also pulled off one of the most epic mergers of all time with Dell + EMC + VMWare 2015-2016 (and which we wrote about here at the time).

    Is there a method to the madness? How does one not just start, but keep, and transform, their company and business? Especially as it adapts to broader, underlying tech platform shifts. Michael Dell shares all this in his upcoming new book, Play Nice But Win: A CEO’s Journey from Founder to Leader… he also, tellingly, may be one of the longest-standing founder-CEOs (37 years so far).

    Because this is really a story about innovation, who decides, who judges, who does it, and where: In the markets, in public, in private; in the both the big picture and the inner detailed workings of a business beyond “cells in a spreadsheet”; and even in fighting — or harnessing! — narratives, whether it’s the demise-of-PC or cloud wars 1.0 /2.0… And where trends like the cost paradox of cloud, and “end of cloud” edge computing, among others like AI & ML, also come in. In this special book-launch episode of the a16z Podcast with Marc Andreessen, Martin Casado, and Sonal Chokshi debate the Cloud Wars to the Company Wars (along with some behind-scenes stories and even some star wars) with Michael Dell… and whether you can really play nice to win.


    image: Dell EMC World 2016/ Dell Inc.

    • Michael Dell

    • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

    • Martin Casado is a general partner at a16z where he invests in enterprise companies. Prior, he was cofounder and CTO of Nicira (acquired by VMware) and is the creator of the software defined networking movement.

    • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

    Uncontrolled Spread: Science, Policy, Institutions, Infrastructure

    Scott Gottlieb, Vineeta Agarwala, Marc Andreessen, and Vijay Pande

      There’s no question technology played a huge role in the recent/current pandemic, including especially in the plug-and-play engineering and incredibly fast development behind the mRNA vaccines… But is there an even bigger role for the private sector, not just government, to play (and partner) when it comes to key infrastructure for future such emergencies, and even beyond?

      Especially given how faulty the translation of institutional science to policy and public health measures turned out to be — for instance, with “6 feet” of social distancing, or with fomite (vs. aerosol) transmission of COVID. And why are we still talking about the same, not specific, vaccine booster for the Delta variant? What can we learn about real-world evidence, other clinical trial approaches, and progressive (vs. binary) EUA approvals when it comes to public health emergencies? Are capabilities like genomic surveillance and mapping strains — which require layers of technology, real time — sitting in the right places?

      In this special book-launch episode of the a16z Podcast, former FDA commissioner Dr. Scott Gottlieb — author of the upcoming new book, Uncontrolled Spread: Why COVID-19 Crushed Us, and How We Can Defeat the Next Pandemic — shares insights on the above, and revealing stories from behind the scenes. Do we need a new entity to manage public health through a national security lens, and is the government capable? Gottlieb debates this and other probing questions from a16z co-founder Marc Andreessen (who famously wrote “It’s time to build“); a16z bio general partner Vineeta Agarwala MD, Phd (who has spoken about the trials of clinical trials, practiced medicine during the pandemic, and more); and founding a16z bio general partner Vijay Pande PhD (who, among other things, founded the distributed computing project Folding@Home which pivoted to COVID proteins).

      One thing’s for sure — with this COVID crisis, we’re at an inflection point between old and new technology — whether it’s in how we make vaccines, or how we apply the fields of synthetic biology and genetic epidemiology in public health response. So now’s the time to look both backward, and forward, to really change things…

      • Scott Gottlieb served as the 23rd Commissioner of the U.S. Food and Drug Administration (FDA) from 2017-2019; is a resident fellow at the American Enterprise Institute, partner at NEA, and serves on several boards.

      • Vineeta Agarwala MD, PhD is a general partner at a16z investing in bio and healthcare technology. She is also a practicing physician and adjunct clinical faculty member at Stanford.

      • Marc Andreessen is a cofounder and general partner at a16z. Marc co-created the highly influential Mosaic internet browser and cofounded Netscape.

      • Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.

      Man, Mosquito, Malaria Vaccine

      Jorge Conde, Rajeev Venkayya, and Sonal Chokshi

      Playing out against the backdrop of a global pandemic (including recent massive surges in regions around the world) is the news that came out a week ago that a candidate “malaria vaccine becomes first to achieve WHO-specified 75% efficacy goal”. While the findings are still in preprint with The Lancet, the resulting buzz and phrases quoted included everything from “unprecedented”, “groundbreaking work”, and “very exciting” to “high expectations”, “highly effective”, and “a hugely significant extra weapon”… A “weapon” in the war against malaria that is — a disease that is estimated to cause over 400,000 deaths each year globally, and predominantly in children under the age of five.

      So in this special 2x explainer episode of 16 Minutes (also running on the a16z Podcast), we —Rajeev Venkayya of Takeda Pharmaceuticals, a16z bio general partner Jorge Conde, and Sonal Chokshi — dig into what’s hype/ what’s real about this news, beyond the headlines and beyond the buzz. What does the data tell us, what does the current study phase mean, and what’s left to get to widespread, real-world use? How does this candidate vaccine (R21 from Jenner Institute/ Oxford University) compare to the other malaria vaccine (RTS,S from GlaxoSmithKline)? How do, and don’t, advances in and around COVID vaccines play here? And why has it been so hard to develop vaccines for this particular disease?

      Because we also cover (as is the premise of the show) where we are on the long arc of innovation… and this is an innovation story that’s been nearly a century in the making.

      Show Notes

      • The urgency of finding a vaccine to combat malaria [2:45] and why malaria is so difficult to treat [5:12]
      • Discussion of the R21 vaccine candidate and its pending study [10:48]
      • Previous vaccine attempts, details around clinical trials [17:48], and the pros and cons of different vaccine approaches [24:27]
      • How vaccine effectiveness is measured [28:54] and possibilities for the future [33:35]


      Sonal: Hi, everyone. Welcome to the a16z Podcast network. I’m Sonal, and this is a special 2x episode of 16 Minutes, which we’re also running on the main a16z Podcast as part of our ongoing such coverage — and it’s all about the recent news, science, technology, problem, and innovations behind malaria vaccines.

      While the discussion includes contrasts and comparisons to COVID vaccines briefly – and also plays out against a broader backdrop of the massive recent surge of cases in India and South Asia, as well as new waves in Brazil, Turkey, France, Argentina and elsewhere — the big news that also came out this past week is that a new candidate malaria vaccine is the first to achieve the World Health Organization’s goal of 75% efficacy, according to the announcement from Oxford University

      Just to give a sense of how big and buzzy this news has been, some of the keywords that have been quoted by experts in many of the releases and articles have included phrases from “unprecedented,” “groundbreaking work,” “very exciting,” to “high expectations,” “highly effective,” and “a hugely significant extra weapon.”

      So, as is the premise of 16 Minutes (which, if you’re not already subscribed to, be sure to find, and follow, in your podcast app), we dig beyond the headlines for what’s hype/what’s real, as well as where we are on the long arc of innovation.

      Our expert guests for this episode are Rajeev Venkayya, President of the Global Vaccine Business Unit at Takeda Pharmaceuticals, where he leads full-stack development of vaccines for tropical diseases like dengue, norovirus, and Zika. He’s also been trained as a medical doctor and served as Director of Vaccine Delivery at the Gates Foundation, and was previously at the White House in biodefense.

      Rajeev has also shared how vaccine development works in general, including outlining the phases and what was accelerated for COVID vaccines in an episode the three of us did last year, with a16z bio general partner, Jorge Conde, who also joins this episode, and has, in fact, been on all of our vaccine episodes. I don’t know if you actually knew that Jorge, that you’ve been on every single vaccine episode. <Jorge: You keep inviting me back!> Yeah, well, sadly — and importantly — we’ve had to cover many different aspects of this topic over the past year and a half.

      And, we’ve covered everything from vaccine nationalism and vaccine hesitancy to vaccine manufacturing and scaling and all about mRNA vaccines and much more. Listeners, you can find all of that at a16z.com/vaccines.

      But for this malaria vaccine, I’d actually love to start by hearing from both of you what your reactions were to the recent news. And I’ll summarize the specifics of the news in a moment, but, would love to just quickly hear the big picture for why this matters, from your vantage point. 

      The fight against malaria

      Rajeev: I think this is a really big deal, because malaria is one of the big three (as we call it) — HIV, tuberculosis, and malaria cause an extraordinary amount of suffering and deaths every year.

      All three of them have proven to be very, very tough targets from the standpoint of vaccine development. There’s no vaccine yet for HIV, despite extraordinary global efforts for decades. We have a vaccine that leaves a lot to be desired in the BCG, that we give at birth for tuberculosis. And there is a malaria vaccine that was brought to the world a few years ago from GSK called RTSS — but, that vaccine, while efficacious, has some room for improvement, and this vaccine potentially could be that improvement we’re looking for.

      Jorge: In addition to what Rajeev said, I would say two of my biggest reactions was number one, the fact that they seem to have early indications of a vaccine that’s highly efficacious against a parasite is no small feat in and of itself (a parasite is a tricky bug).

      And the second one I would just point out is, you know, obviously, vaccines have gotten an incredible amount of attention over the last year or so, given the COVID pandemic. But this vaccine, first of all, has been decades in the making… <Sonal: Nearly a century, I heard!> …nearly a century of effort to try to find a vaccine against this parasite, and this breakthrough didn’t come from the same technology that gave us the COVID vaccine breakthroughs. And to me, what’s fantastic to see is, the dividends that come from decades of research that pay off using, sort of, “old-world” technology. What I mean is, you know, this was a vaccine developed using more traditional vaccine production methods, versus what we saw in the course of one short year with the mRNA vaccines that BioNTech and Pfizer and Moderna brought us for the COVID pandemic.

      Rajeev: Yeah I think for a long time to come, we’re gonna think about vaccines in terms of “pre-mRNA” and “post-mRNA.”

      This is technology that was developed well before mRNA vaccines came of age. The technology that was used is actually proven — it’s been used in the hepatitis B vaccine, as well as the human papillomavirus… <Sonal: HPV, yeah> …yeah, these virus-like particles that are very, very effective at “presenting antigen” to the immune system so that it can recognize it, and then develop an antibody and cell-mediated immune response to that antigen.

      The challenge of finding a vaccine

      Sonal: Well, let’s dig into what this vaccine is and how they work. But first, just to put this whole problem in scope, just quick statistics. Over 229 million cases of clinical malaria were reported the year before [2019], and the World Health Organization estimated that malaria causes over 400,000 deaths each year, globally. I mean this seems to be concentrated in Africa and South Asia, Southeast Asia, different regions, but it is important to just note the scope of the disease.

      Jorge: A lot of these tropical diseases also happen to be “poor country diseases.” I have to wonder if this tropical disease was endemic in the richer parts of the world — had we, you know, gotten to this breakthrough sooner.

      Sonal: I was wondering the exact same thing, Jorge. The fact that we got to a COVID mRNA vaccine in less than a year, versus this disease that people have been working on for nearly a century — it almost makes you wonder, like, if this were prevalent in the United States, we would have solved this like 20 years ago.

      Rajeev: Possibly, possibly. There would have been a lot more R&D for sure.

      Sonal: I frankly think this is also true for women’s health, but I will go on that rant later.

      Jorge: I was about to say that.

      Rajeev: Yeah. It’s important to note that almost all the deaths are happening in sub-Saharan Africa, and two-thirds of those deaths are in children under the age of five. This is a very, very significant global burden of disease that desperately needs a safe and effective vaccine.

      Jorge: What some people don’t realize is, not only is the disease burden high, but malaria is something you can get infected with again and again and again. You don’t get sick once and then you’re immune. There are reinfections. And so that just adds to the burden.

      Sonal: I’m really glad you’re bringing up the point that it’s not a one-and-done disease, like another type of disease. I actually read a statistic that the average in a lifetime is six times, that a person can get malaria six times in their lifetime.

      Jorge: At some point, you do develop immunity. Like, the reason why people catch malaria an average of six times and not an average of the number of years that they’ve lived — because eventually you become more resistant to it.

      Sonal: Right. It’s like my parents, they never get stung or sick. But when we were kids, my brothers and I used to count and compare our mosquito bites, and we’d have, like, competitions for, like, how many we had. I would have (I’m not exaggerating), hundreds of bites on my body. And we did everything by the way. Nets, the coils, everything, you name it. The anti-malarial drugs — although I didn’t take mine, which is why I got sick. I was a young kid. I spit mine out. It was disgusting, and no one watched to see if I swallowed it.

      Jorge: And by the way, the way that these anti-malarial pills work (at least some of them), is they essentially poison your blood so that the parasite…

      Sonal: They taste poisonous. They taste like poison.

      Jorge: Yeah, the parasite can’t survive in your bloodstream.

      Sonal: So, why has this disease been so difficult? And in general, is there a difference when you’re designing a vaccine to target a parasite versus a bacteria or other viruses? The reason I’m asking is because, Jorge, you mentioned the malaria parasite — it has a complex life cycle, and can mutate. <Jorge: It’s wild> It’s crazy. I mean I’ve experienced a very mild form of it when I was a child, and just a little anecdotal bit of detail (that I don’t know if people who have never experienced this would ever know this), but in my experience, I had like the highest fever one day; the next day, it was as if I were a completely healthy person. And then the third day, I had a super high fever again. I don’t know if that’s a normal thing, but that was bizarre to me. Like, I had no idea why that even happened.

      Jorge: Well, part of the reason why that probably happened is this sort of funky lifecycle of the parasite. So, you know, an individual gets infected when they are bitten by a female mosquito that’s carrying an infectious form of the parasite. That goes to your liver, where it continues to reproduce. Then it gets released to your bloodstream, where it attacks your red blood cells. And, it replicates within your red blood cells, and, when your red blood cells get too full of parasite, they burst, release a bunch of parasites into your bloodstream — that’s what causes the fever to spike. And then, it gets tamped down, and then when there’s another burst (of another set of red blood cells), the fever spikes again.

      And then, to complete the life cycle, you now have a sort of premature version of the parasite floating around in your bloodstream, and you get bitten by a mosquito — and now that goes back up into the mosquito (to complete its sexual maturation). So it actually comes full circle. A lot of people think about parasites having a host. You know, in the case of malaria, malaria is being raised in sort of like shared custody between man and mosquito.

      Sonal: That’s an incredible explanation.

      Do you have any thoughts specifically on what it means to target malaria, as a disease?

      Rajeev: Well, it’s almost a self-fulfilling situation, because the fact that a person can have malaria multiple times tells us that the immune system is having a tough time with this parasite. The immune system is not able to identify the parts of the parasite that it can then attack when it gets reinfected to prevent the illness from recurring.

      And so, if the immune system, which is super sophisticated, is not able to do that, then almost by definition, it’s going to be a tough vaccine problem. One of the reasons is, the parasite can be quite effective at evading the immune system in the way that it grows in a person’s body. The parasite’s life cycle involves transmission through a mosquito that bites a person who gets infected — but then once that parasite is in a person, it has multiple stages of its growth that can be difficult to target. And so, that’s another unique feature of malaria.

      A promising vaccine candidate

      Sonal: So, let’s talk specifically about this vaccine. To summarize, the candidate vaccine here, it’s called R21. It was developed by scientists at the Jenner Institute at Oxford. And the reported findings are that they demonstrate a high-level efficacy of 77% over 12 months of follow-up, in a study with African children. Specifically, 450 participants aged 5-17 months, all from the country of Burkina Faso. Most of the doses were administered before the peak malaria season there (three vaccinations were administered at four-week intervals), and then a fourth dose came one year later.

      So, that’s a super high-level summary. One more quick note, this was all part of the phase 2b trial — randomized, controlled, double-blind — the findings are still in press with the medical journal The Lancet (and I will include all links and sources that are mentioned in the show notes as always).

      Rajeev: One thing I do want listeners to know is that the data we’re discussing today and you’ve seen in the media comes out of a preprint — which means this is not yet a peer-reviewed publication. Other experts will look at the study design and the results and they’ll ask critical questions of the researchers that they’ll then have to address in their responses, ultimately resulting in a peer-reviewed publication. Now of course we do hope that the essence of the findings will remain unchanged, and that the conclusions will largely be the same — but we can’t say that for sure until the peer review process is actually concluded.

      Sonal: I mean, the preprint — lately “science-by-press release” is definitely a thing that’s accelerated in the last year for sure. I’ve seen it at a whole new scale that I’ve never seen before.

      Rajeev: Yeah, and you know some would say that given that we’re living in COVID times, where literally every day matters in science, that we have to accept science-by-press release, hopefully followed very quickly by peer-reviewed publications. But the peer review and publishing process just takes too long, frankly, for COVID. And so we often end up having to rely on press releases and preprints.

      But, I do want to point out that the preprint came out right around World Malaria Day, which is April 25th. And I think that’s probably why these guys released the preprint when they did.

      Jorge: Wait, it’s April 25th you said? <Rajeev: Yeah> So World Malaria Day is the same day as DNA Day. That’s interesting.

      Sonal: Oh, that is interesting!

      Rajeev: I didn’t realize we observed DNA Day. That’s…

      Jorge: Yeah, because April 25th was when the Nature article [was published] that Rosalind Franklin published her Photo 51, and Watson and Crick published their one-page, structure-of-DNA paper. And then, in an act of symmetry, the tie between Francis Collins and Craig Venter in sequencing the human genome was announced by Bill Clinton on April 25th.

      Rajeev: Oh my gosh, I didn’t realize we were — there was a World DNA Day. Thanks for sharing that.

      Sonal: So, back to the point about malaria. Anything more to say on the specifics of malaria as a disease that’s relevant here?

      Rajeev: Well, there are a couple of things to think about when you are looking at vaccine development. And one reason, and perhaps the most important, is to prevent the severe illness that comes from it. But there is another objective with malaria vaccines, which is blocking transmission. Now, we’ve all heard with COVID, that one of the goals of vaccines is to reduce transmission in the community to help us get a handle on the pandemic. The same concept applies when it comes to malaria, but the vaccine approach is very different.

      And — this will blow your mind — when we think about blocking transmission of malaria, the way it’s being approached from a vaccine standpoint, is to prevent a mosquito from picking up malaria from a person that has it. And the way you do that is by designing a vaccine that will generate antibodies that are taken up by the mosquito, along with the parasites, and preventing the parasite from reproducing inside the mosquito.

      So, you’re giving a person a vaccine that’s not going to prevent their illness, it’s going to prevent the parasite from reproducing in the mosquito, so you’re actually indirectly vaccinating the mosquito (and not the person), because that person could still get malaria illness.

      Jorge: The fact that we’re vaccinating mosquitoes, I think is wild.

      Sonal: That’s fascinating. Let’s actually talk about the findings.

      Rajeev: So, basically what the researchers did is, they took this group of children that are in a place with a very, very high incidence of malaria, right before the malaria season, and they gave them three doses of vaccine with another dose a year later.

      They then counted cases of malaria, so they monitored for fevers in the children, and when a child came in with fever, they would do a set of diagnostic studies, and if they were found to have malaria, then they would be classified as being a malaria case. And then you compare across the three groups, and the three groups were a control group that received the rabies vaccine, there was a low dose adjuvant group, and a high dose adjuvant group. And both of those had the same amount of the protein that makes up the vaccine.

      Sonal: Can you quickly explain what an adjuvant is and why it matters, because everyone always mentions that and how that plays in with the way the vaccine works.

      Rajeev: Adjuvants are what you might consider immune boosters. And so, for any given amount of let’s say protein that you’re giving somebody to train their immune system to recognize a virus or bacteria, or in this case, a parasite, you can get away with a smaller dose of that protein if you give somebody an immune-boosting adjuvant.

      The adjuvant that is used in this trial is the same adjuvant that a company called Novavax is using for its COVID-19 vaccine. And this is an adjuvant that is chemically related to an adjuvant that is used by GSK in their vaccine against shingles that is currently on the market.

      So, it’s an adjuvant that has been proven (at least in that vaccine) to be very efficacious. And at least based on the phase two data that we have, with the combination with this — what we call “virus-like particle” — it also appears to be quite effective at generating a protective immune response.

      Jorge: I understand why you’d have a high dose and a low dose arm. Why is the control arm a rabies vaccine versus saline?

      Sonal: Yes, I was wondering.

      Rajeev: Yeah, well, you know, when you’re looking at the immediate safety of a vaccine — meaning sore arms, fevers, or chills that you might get after a vaccine — you’re gonna see that to some degree with many vaccines. And so, we want to do a “fair or appropriate comparison.”. And so, using another licensed vaccine that would be appropriate for the population that’s in the study is an approach that’s often taken to have the control group even out between.

      Previous attempts and clinical trials

      Sonal: I see. You’re sort of controlling for the variables you’re trying to measure. <Rajeev: That’s right> That makes a lot of sense. So, why — this is the real question here, the big-picture question — why has it been so hard to develop a vaccine for malaria? Now, we’ve talked already about the difficulties of the disease, but, like, if you look at the arc of it — Rajeev you said there’s one vaccine I think right, the GSK one, RTSS, and I think the last thing that I read was that they demonstrated 55.8% efficacy in African children.

      Can you explain that vaccine, and tell us more about — I’m really trying to dig into, like, why it’s been so damn hard to actually get here, and why this milestone is so significant.

      Rajeev: Well, it was a big deal when GSK showed that their malaria vaccine worked a few years ago in a phase three trial. Now they showed that initially they had about 56% efficacy in the first year. Unfortunately, that efficacy wore off over time. And so, if you looked at how efficacious it was after four years, it dropped down to 36%.

      And so, that vaccine is not yet widely recommended. There’s a pilot program rolling out in a handful of countries that’s happening as we speak.

      Sonal: And by the way when you describe the pilot program, you were involved with that organization, Gavi (I believe) is the one that’s sort of helping the World Health Organization pilot the GSK vaccine in Kenya, Ghana, Malawi, I believe.

      Rajeev: Yeah previously, I served on the Gavi board, which is the primary financing entity for vaccines for low-income countries. The data coming out of that program will inform the future implementation of that vaccine. Based on that level of efficacy that was seen with that first malaria vaccine, the WHO later came out with a target efficacy of 75% for future malaria vaccines. That was what presumably these researchers were going for when they tested their vaccine, and they were actually able to hit that target.

      Sonal: So, the GlaxoSmithKline vaccine has been through lots of clinical trials — a lot more clinical trials than this one has — and this particular vaccine, going back to this specific news, of the Oxford vaccine news. This is a phase 2B trial, it’s not phase three yet. Can you quickly explain what it means to be in phase 2b and what comes next in phase three?

      Rajeev: Sure, sure. So when we take vaccines through clinical trials in people, we start out with the studies to assess the safety of the vaccine to pick up any significant problems in small numbers of people before you go into bigger trials. And that’s done typically in phase one. In those phase I trials, you’re also assessing what the right dose of the vaccine could be. So you might have high, medium, and low doses of the vaccine to give you a sense as to what the right dose is to take it to further clinical development.

      In phase II development, you’re often going into [a] larger number of individuals, and confirming that you are at the right dose. You might even still have different dose levels in your phase two. In a standard phase two trial for infectious diseases, you’re not actually looking to see whether you’re preventing infection, because there’s a relatively small number of people in phase two trials, and that wouldn’t be enough to actually be able to statistically measure a difference in the vaccine group versus the control group.

      A phase 2B trial is a little bit different: You have an even larger trial than a standard phase two, and these are often powered statistically to give you a sense as to whether the vaccine is actually working. This is a way that a company might de-risk the vaccine program before they go into a very large, very expensive, sometimes very long phase-three clinical trial.

      So, this is that type of trial, where given the high incidence of malaria in this region and this population, they were actually able to show whether or not the vaccine works at preventing malaria.

      The next step after this would be a large phase III trial. We can benchmark against GSK’s phase three trial of their malaria vaccine, which had about 15,000 children in it. And as we all know from COVID, the phase three clinical trials there have ranged from 20,000 to 60,000 individuals in any given phase three. So, this is smaller than that. And the main reason they were able to get away with a smaller trial is because the incidence of malaria was so high in the places where they were testing the vaccine.

      Sonal: And for specifics, what I understand is that the recruitment of the phase three trial has already started, and they’re recruiting 4,800 children aged 5-36 months across four African countries. But here’s a little twist. So, you mentioned Novavax — so they’re one of the partners that’s collaborating with Jenner. But the other is, of course, the dominant player, the Serum Institute of India. They are obviously going through a massive COVID — so they’re actually delaying this a little bit, from what I understand.

      Rajeev: Yeah, you know, it’s worth talking a little bit about that comparison between this vaccine and the GSK vaccine. Actually, they’re quite similar. They use similar adjuvants or immune boosters. The adjuvant used in the GSK vaccine is their proprietary adjuvant called AS01. In the case of the Oxford vaccine, it’s called Matrix-M, which is the Novavax vaccine. They’re both derived from tree bark (believe it or not) and so the chemical construct between the two, of the adjuvants, is quite similar.

      The other parallel between these two vaccines is the virus-like particle approach is the same one taken between the two. They both use the hepatitis B surface antigen, which forms the core (or the base particle) for the vaccine.

      The primary difference is that there is less of the hepatitis B surface antigen protein in the new Oxford vaccine. So there is proportionally much more of the malaria protein on the particle, than there is in the earlier GSK vaccine. And that is thought to be (potentially) a contributor to the difference that we’re seeing in efficacy with this vaccine, versus what GSK saw.

      Sonal: And to be clear — I just want to kind of take the bottom line on that. Basically, what both of these vaccines are doing is targeting the parasite in that sporozoite phase of the lifecycle, which is when it enters the human body from the mosquito. And, the vaccines you’re saying — the main difference between the two vaccines — the R21 includes a higher concentration than the GSK. But that’s the primary difference. Other than that, they’re relatively similar underlying mechanisms.

      Rajeev: That’s right, these two vaccines are quite similar insofar as they use a similar adjuvant, and they also have a similar structure.

      Different vaccine approaches

      Jorge: You know, in the case of this malaria vaccine, when we look at SARS-CoV-2/COVID as an example, the vaccine makers all sort of thought the spike protein was the best target or the most likely target, and that’s where the major players focused. Is there a similar consensus in malaria as to what the right targets are, or has that in and of itself been an odyssey?

      Rajeev: Great question. There’s a lot of consensus around what’s called the circumsporozoite protein, let’s just say it’s CSP — which is the protein that is used in both the GSK RTSS vaccine as well as the Oxford R21 vaccine that we’re talking about today. And given that we’ve had limited efficacy with the vaccines against CSP — it’s not to say that something else won’t turn out to be better in future vaccine clinical trials.

      Sonal: What happens if the bets we’re making with these vaccines — and this, I think, is the underlying thrust of Jorge’s question — is that it’s not actually effective at that sporozoite phase, do you have any thoughts on that? When we put all our bets on the coronavirus that hey we’re gonna focus on the spike protein, we’re taking a good bet — and so far it seems to have borne out, given that even with the new strains, that it’s still targeting the spike.

      Rajeev: Well, it’s possible that we won’t be able to get there with this protein as the primary target, and that we would have to perhaps add a second target to the vaccine in the future. And it’s possible that we would have to add a second protein, or go after a different protein, in order to get efficacy against the parasite in this stage of its lifecycle. There are a lot of other proteins that one could target on this parasite.

      Sonal: Got it. And actually, quick question for Jorge — is it naive of me to ask whether an mRNA approach would be more efficient? You guys put it really well, like, the old world/the traditional world versus a new world we’re in. Would a different approach to vaccines do a much better job? Because when I think of how we are thinking of these new batches of vaccines that we have in our bodies as, like, “software as a service,” are we able to do more with that — like, is that even on the horizon for malaria, or is that like just a pipe dream?

      Jorge: I don’t think it’s a naïve question at all. You know, I — the vaccine producers that have mRNA technologies are looking at a broad range of infectious diseases for their next areas of focus. So, influenza is — you know, the flu is going to be an area of focus. The common cold is potentially, you know, on the table. So, viruses like that, clearly are sort of a next-horizon focus for these companies producing mRNA-based vaccines, and I don’t think it’s unreasonable to assume that their focus will expand beyond that.

      I don’t know, technically, if an mRNA-based vaccine against malaria is feasible, but if you’re looking at surface proteins, arguably, you could theoretically develop an mRNA vaccine against the malaria parasite.

      Rajeev: I agree with what Jorge said. I don’t think we can say that just because mRNA is a new technology that it’s more likely to be effective against malaria; however, I do think we can say that we need to give it a shot, because there’s so many advantages of mRNA approaches relative to more traditional approaches of developing vaccines.

      One of the really interesting possibilities is that you could combine vaccine approaches into a single mRNA product. So, for example, you could have a CSP part of the sequence in your mRNA vaccine — which is the same protein that is targeted by these first two malaria vaccines that we’ve been discussing. And then you could have a second sequence or set of sequences that are targeting that sexual form of the parasite that the mosquito takes up when it has a blood meal, and then goes on to transmit the parasite to somebody else.

      Remember, one thing that we need to realize is that second vaccine target where you’re trying to prevent the mosquito from going on to transmit the parasite — a vaccine like that would not prevent the actual illness associated with malaria. So, if you had a vaccine that was just focused on that, you’d be giving it to somebody and telling them “Look, this isn’t going to keep you from getting sick from malaria, it’s going to keep you from passing the malaria parasite onto somebody else.” That’s not a very attractive vaccine for someone to take. But if you combine that with something that also prevents them from getting sick in the first place — which could be this other part of the mRNA vaccine — then you’ve got a vaccine that everyone’s gonna want to take, and helps us to reduce malaria transmission, and maybe even eliminate malaria long term.

      Measuring vaccine effectiveness

      Sonal: Wow. Okay. In general, for context, over 100 malaria vaccine candidates have entered clinical trials in the past decades. But none has shown this level of efficacy that’s been targeted by the World Health Organization.

      Now, again, to be clear, we’re talking about phase 2B. It hasn’t done large scale yet. But, one of the people who heads the World Health Organization malaria vaccine implementation program, argues that even modest efficacy would have a high impact precisely for the reason that people get the disease over and over again.

      So, can we quickly talk about what the numbers of efficacy mean? They evaluated the vaccine safety, efficacy, etc., over one year — and what it means in, like, real-world practice?

      Rajeev: Yeah, let me touch on the three parameters of vaccine performance that we often look at. One is immunogenicity. This is the easiest to measure. It’s simply the antibody response to the vaccine. We measure antibody levels in the bloodstream, and that becomes a measure of what we call immunogenicity. The reason that’s important is because the level of antibodies you generate with a vaccine might correlate to that vaccine’s ability to protect you from getting infected or contracting the illness associated with that infectious disease.

      A second term we use in late-stage clinical trials is efficacy. And what that typically means or measures, is the ability of a vaccine to prevent the illness associated with an infectious disease. That term “efficacy” is very specific to the context of a clinical trial. So, there is usually a point estimate (let’s say 70% efficacious), and it’ll have a confidence bound around that, which represents kind of the error range given the size of the sample of your study.

      The third measure we often talk about is effectiveness. Another way to think of this is real-world effectiveness. So, this is the assessment of how well the vaccine functions outside of a clinical trial, when you’re in the real world. You’ve launched the vaccine into a population (like we have with our COVID vaccines), and now we’re measuring how much illness and disease there is in a population.

      Outside of the very controlled environment of a phase three clinical trial, where you may be telling people to do a number of things in order to protect themselves from the infectious disease. And so, you might see that a vaccine performs very well in the context of a phase three clinical trial. Then once you actually roll it out to the population, it doesn’t perform quite as well because these are real-world circumstances where people aren’t doing all the same things they wouldn’t be in the context of a clinical trial.

      Sonal: So is there anything important to note about the question of effectiveness when it comes to this vaccine? Now again, it’s not in deployment yet, etc., but what are the considerations. One that of course comes to mind here is like, there’s like multiple doses.

      Jorge: Multiple doses, yeah, I think that’s the biggest one.

      Rajeev: That’s a big one. Multiple-dose is a big one. But there’s also, you know, in the clinical trial, people may have been very good about using bed nets at nighttime at home to prevent mosquitoes from biting. But in the real world, they may not keep up with their bed net use. They may not keep up with their indoor residual spraying of insecticides, which is often used to reduce biting in mosquitoes.

      So, there are a variety of things that could increase the likelihood of getting infected in a real-world setting, which could correspondingly reduce your measured effectiveness.

      Sonal: So, the other promise of this candidate vaccine from Oxford, is also the potential to get more high-volume, affordable vaccines. I read that it’s easier to make than the one that’s being used with the GlaxoSmithKline one, suggesting that it could be cheaper. Do you know if that’s true or not?

      Rajeev: I’m not aware of the differences in the cost of making the two, because they’re quite similar. But it is using a pretty standard method of making vaccines. It’s manufactured in yeast (which is a tried-and-true way of manufacturing protein) — and the fact that Serum Institute of India has done this, means almost best-in-class in terms of efficiency, plus very high quality of vaccine manufacturing, given Serum Institute of India’s great track record here.

      So, that’s also good news because this is a vaccine that is going to have to be priced at a level that is affordable for the poorest countries in the world, and can be purchased by Gavi. The fact that the cost structure is likely to be very low, helps to ensure that we’ll have a low price for this vaccine (at least for poor countries).

      Possible next steps

      Sonal: There hasn’t been a EUA for a vaccine in malaria, especially given the fact that malaria kills more people in Africa than COVID does currently. Do you think it’s possible that they might do some kind of EUA-type of situation for this?

      Rajeev: I don’t think so. I would expect them to go through a standard (although accelerated) review process. One of the things that you are able to do in a standard review process is make sure that the manufacturing processes are all very well worked out, and validated, and reproducible with high quality.

      It’s important for any vaccine, but certainly a vaccine like this that’s going to be going into vulnerable infants, you want to make sure that you do absolutely everything. And hopefully in the fastest time period possible.

      Sonal: Right. I mean, we don’t even have a vaccine for kids for COVID yet, in fact, right, so.

      Rajeev: That’s right, that’s right. One of the things that may be coming out of COVID is that we’re all a lot more attentive to global health problems that affect everybody in the world.

      Jorge: I have to believe that we’ve also now developed capability, capacity, and political will in terms of vaccine production to do these things at the scale and speed necessary to hopefully benefit the entire world. My hope is that not only have we developed new technologies — you know we are now in a post-mRNA vaccine world (as Rajeev mentioned) — that have the potential to be um pointed at other infectious diseases, and hopefully give us other future breakthroughs.

      When it comes to malaria, we’ve been pointing all of our guns at this for a long time. So, we’ve not only been looking for a vaccine, but as folks know, there have been philanthropic efforts to get bed nets out there. There’s of course tons of efforts in terms of insecticides, to reduce the population of mosquitoes. And there’s even, you know, engineering biology approaches to create genetically modified mosquitoes that are resistant to malaria. So, this is one weapon in what is a pretty deep armament to try to beat this thing.

      Rajeev: Yeah, it’s absolutely true that this could be a critical tool in our toolkit. I look forward to seeing the peer-reviewed publication and hope that we’ll be seeing just as good results when the vaccine goes into phase three.

      It’s also exciting to think about the end game. There is a day when we could imagine eliminating malaria from many more parts of the world, and possibly even eradicating it from the face of the Earth. Now, that’s not going to be easy, as smallpox and polio have proven (and certainly malaria is very different from those other diseases which are caused by viruses) — but it is something we can hope for.

      Jorge: I was gonna say, we know the date of eradication will be April 25th, we just don’t know what year.

      Sonal: Thank you so much, you guys, for joining this week’s episode of 16 Minutes.

      Rajeev: Thanks a lot for having me, Sonal.

      Jorge: Thank you, Sonal. Thank you, Rajeev.

      • Jorge Conde is a general partner at Andreessen Horowitz where he invests in companies at the cross-section of biology, computer science, engineering. Before a16z bio, he was CSO at Syros, cofounded Knome, & more.

      • Rajeev Venkayya

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      The Creator Economy — NFTs and Beyond

      Kevin Chou, Jesse Walden, and Chris Dixon

      In today’s episode of the a16z Podcast, we’re talking about the Creator Economy, and how NFTs (but not just NFTs!) are making it possible for artists, musicians, videogamers, game developers, and writers to create entirely new markets to make money from their work and engage with their fans.

      Part of this emerging picture is social tokens, which share a crypto foundation with NFTs, but unlike NFTs (which are non-fungible tokens, in which each token is unique), social tokens are typically fungible, meaning each token has the same value. (Listen to our explainer episode “All About NFTs” with Sonal Chokshi, Jesse Walden, and Linda Xie, or see our curated NFT Canon for much more info on NFTs!)

      This hallway-style chat features a16z General Partner and crypto investor Chris Dixon, talking with Kevin Chou, who founded Kabam, and is the founder of Rally, an open network on Ethereum where creators can launch social tokens; and Jesse Walden, the founder of Mediachain, a music attribution protocol that was acquired by Spotify; he’s now the founder of crypto venture fund Variant.

      They’ll talk about how musicians, artists, and writers can think about NFTs and social tokens as well, and how those different types of assets can interact to create models that haven’t existed before.

      Chris starts off the discussion by talking about the emergence of crypto tokens, and a look at how videogames and gamers were early to the idea of community engagement and digital assets, and how that model is beginning to spread outward.

      Show Notes

      • The growth of NFTs from gaming tokens [1:20] and how they might be used in the real world [6:31]
      • Issues of portability and security [11:56] and the fractionalization of NFTs [16:16]
      • How NFTs and fungible tokens may affect music creators, including relating with fans and ownership rights [19:46]
      • Impact of NFTs on writers and how digital tokens could be used to fund new ventures [23:49]
      • Social tokens and NFTs as part of the same economy [26:27]
      • The role of big tech platforms [28:16]


      Expansion of NFTs and gaming tokens

      Chris: We’ve all…all three of us and our friends have talked about this stuff for years. I think we’re starting, this year, to see what you can kind of call app layer mainstream kind of crypto token things happening.

      And so, that, of course, is a lot of people have heard about or, you know, originally, I think it starts with NFTs, so non-fungible tokens. But Kevin, you’re working on, is sort of the fungible token counterpart to that, which are tokens that would be associated with communities on the internet. I kind of think of it as analogous to how modern video gaming works, where you have, a game like Fortnite, and the most progressive games. The game themselves are free, but you have in-game currency. In the case of Fortnite it’s called V-Bucks. And then you use that currency for various things, including for buying digital goods, like, in the case of Fortnite skins, and emotes, and things, which, you know, in the web world, the V-Bucks would correspond to social tokens, and the virtual goods to NFTs, right?

      And so I think what I believe is sort of happening now is that video games, which are the most advanced in thinking about how to engage people in social software, and in a way that both goes viral and spreads on the internet, but also makes them money. Those ideas that have been developed over the last 10 years in the gaming world are now propagating out to the rest of the internet, in the open internet. And that, of course, is going to have some similarities, including a lot of design overlap, I think, but also differences, in the sense that these crypto blockchain concepts exist on the open internet and not within silos.

      That’s where I feel like we are. And I think the first bit of the kind of sunlight has broken through in the NFTs and people are starting to see it, but there’s a whole bunch more hopefully coming in the near future.

      Kevin, maybe you could describe how you think that might evolve over the next year or two.

      Kevin: The gaming world is a little bit unique because we created these online communities that had a deeply integrated, set of social interactions and communication tools.

      And so, you know, before there was social media, there were these games that these nerds like me kind of hung out in, and we developed our friends and communities, and, played these games. And we cared about what our mounts looked like, we cared about what our skin looked like, and how we appeared to the rest of the community that we developed relationships with. And, you know, today, you don’t need a game for that. You don’t need a World of Warcraft, or EverQuest, or something like that.

      Now you have Twitter, you’ve got Facebook, you’ve got Snapchat, you’ve got TikTok. The game is not just in the game anymore, it’s happening across all of social media. It’s happening across forums, and Reddits. I mean, it’s happening everywhere.

      And if we could build at the blockchain layer, how do we then take that and propagate it all across the internet, and not just have these things be in games?

      Chris: Yeah. I mean, Jesse, you have a long background in music. So, like, the video game industry, I think, is something on the order of $150 billion, with a B, per year in revenue. And I believe the music industry is something more like 20, and for the most part it’s not really grown with the internet. And why isn’t a musician, with a community on YouTube, or Twitch, or some other place, just kind of an MMO. Instead of shooting other cartoon characters, you’re listening to music and talking to people. You know, and why can’t they take advantage of all the same monetization and engagement technologies that the gaming world has developed? Certainly, there’s no lack of passion for music, you know. People are just as passionate in those environments, arguably, more so than in games.

      Jesse: Yeah, well the sort of recorded music industry hasn’t grown all that much in the internet age, one thing that’s interesting is, like, the live music industry has grown. People are definitely engaged with musicians.

      Chris: Well, but what they’re doing is, that’s the virtual…they’re monetizing the complement, right? So they’re using the internet as the free part, and the offline is the premium, right? To analogize, to, like, productivity software and freemium models. But there’s no reason they couldn’t have a scarce resource on the digital side as well, right?

      Jesse: And I think that’s what’s been missing today, is that, you know, music became free with the advent of mp3s and piracy. And then Spotify, sort of, you know, won by making accessibility super convenient. But with that, you also sort of demolish the value in the sort of scarce creative work that artists are producing. And I think NFTs have reintroduced, you know, the concept of scarcity to the digital realm, and sort of given fans a new way to patronize creators, express their support, route financial value to things that they want to see in the world. And from there, there’s all kinds of interesting things you can do with them, and communities can build around them, and start to get more into social tokens and the like.

      So, one analogy I think is useful when thinking about social tokens and music is, you know, artists had fan clubs, right? And if you bought a membership into a fan club, you got maybe access to the artist’s meet-and-greet, you know, backstage or something like that. But, you know, we haven’t had a digital equivalent of that today. And I think social tokens might be it. You’re becoming a member of an artist or creator’s sort of community by owning their token. And that probably gets you access to all kinds of new cool experiences.

      In crypto world, a lot of people talk about how early they bought Bitcoin, right? It’s sort of a signal of, like, you know, how O.G. you are or how deep you are in the space. And I think that same behavior definitely exists in, you know, certainly with indie music fans being early to a band or whatever. Now you can prove it and profit from it, which is cool.

      Real-world applications

      Chris: So, Kevin, what does this mean in practice? Can you just kind of walk through what the user interaction is? When will people encounter this? And what will their experience be?

      Kevin: Yeah. I think there’s a few different dimensions So one is how easy is it for the average fan, not the crypto audience but the fan, to figure out how to earn, or buy, or trade either an NFT or a social token. And there’s a lot of different approaches there. There’s very crypto-native type of approaches, where everything has to be on chain. It’s got to be held in a non-custodial wallet, etc., to be considered real. And on the other side of it, what we’re trying to experiment with is how do we make that onramp experience for the average fan pretty simple. Vertically integrate as many different things as possible, have the first time crypto experience for the average fan be something that they would expect from another internet service. Some other approaches like what Zora and others have expressed as kind of putting up a big enough barrier so that the fans will have to figure it out.

      So there’s all sorts of different approaches, and there’s no right or wrong answer. And different young musicians or celebrities will figure out what’s best for their fan community and do that.

      The second dimension is, once you own this thing, how do you actually use it? So it’s great if I have an NFT in my wallet. Okay, maybe I can show people a link to my MetaMask wallet, and people can look at the NFTs that I have in there. But how do I actually show this thing to the rest of the community that cares about it.

      And so there’s a lot of work in terms of just status and reputation, and being able to show off different things. But I think, even more importantly, is how do you actually potentially use these NFTs. There is no doubt in my mind that five years from now, maybe even sooner, your backstage pass will literally be an NFT. Somebody will stop you at the velvet rope, they’ll scan your QR code that shows that you indeed own the NFT.

      Chris: An NFT, will it just be like  a digital equivalent of a backstage pass? I find one of the interesting things about NFTs is that it can be multipurpose. Like, it could be both a beautiful picture and a backstage pass, and an investment opportunity, right?

      Kevin: There’s nothing that says that an NFT can only have one use case, right? Certainly, you’re talking about a financial or economic dimension of the NFT having value of whatever the community or the fan gives it, or what the next highest fan would pay for it. But then you can use it as a backstage pass after the event, right? It’s not like the NFT disappears. I mean, you could certainly configure it that way, if you’re a musician, and say, “Hey, once you use the NFT, it burns.” You could certainly do that. But in this particular case of a backstage pass, what probably makes more sense is that you still own it as a fan. And a year from now, five years from now, it’s proof that I went to this event and I was a fan from five years ago when the band was still undiscovered, or whatever it was. So the NFT could be a collectible.

      A lot of what’s happened in the gaming realm, for example, is creating sets and creating different ways that you can compose different items together. So in the MMO world, one of the primary mechanics that have evolved is taking, you’ve got to get this leather strap, you’ve got to get this gem, you’ve got to get this catalyst, and you’ve got to go get this ticket. And you put all of them together and it gives you this new thing, right? And so what happens if you own a track from the musician, what happens if you then combine that with one of the backstage passes that show you’ve been to an event? And then you combine that with something else that shows that you bought a vinyl or equity.

      So we’re going to see all sorts of different ways you could create NFTs. You can use them, you can then as a creator say, hey, if you go collect a bunch of these other things, you can then forge a new type of thing that you only get by being a true fan of mine.

      Portability and security issues

      Jesse: I think the key thing that, Chris and Kevin, you’re both touching on is that these assets are programmable, right? And you can sort of compose them into all kinds of new use cases, and they’re also portable, and that’s because you own them in the same way you own Bitcoin. It’s yours, you can choose to park it somewhere like Coinbase or you can take it with you to another platform. And so, because they’re both programmable and portable, you can take your assets and bring them into all kinds of new experiences that developers build, that give them different utility. Like, it can be a fan club backstage access pass, but a third party developer can add some additional functionality to an NFT or social token that makes it useful in another context for a different purpose.

      Chris: I think when people start to really get a tangible feel for it, it will make a big difference. So, right now, if you’re buying a piece of art on Foundation or something, or you’re buying a basketball moment on Top Shot, you basically can use it in that context. But because they’re blockchain objects that are portable, third parties will start creating experiences around them. I think you’ll very soon see companies that get funded, that let you do games, and social experiences, and other things with all of these assets.

      And kind of the broader thing is you’re inverting the polarity. The earlier web was built around applications. The next, the web3, will be built around these user-controlled objects, as primary and then the applications come secondary, and serve them.

      Kevin: We’re talking about Flow and NBA Top Shots just absolutely exploding. And then we have OpenSea, and Zora, and a few others on Ethereum, and then there’s emerging some of these Layer 2 NFT, you know, sort of like purpose-built Layer 2s for NFTs and a few other things. And I wonder about this portability and kind of what you guys see as how portability evolves over the next year or two, as this fragmentation of the Layer 1 and Layer 2, things fragment more and more.

      Jesse: So, for social tokens, I think there’s going to be a lot more interoperability because they’re fungible tokens, they’re sort of easier to port around, and it’s okay that they sort of fragment across the universe of various blockchains. NFTs, I struggle a little bit more to reason about because one thing that makes an NFT valuable is the fact that it’s unique, it’s scarce, and therefore its provenance is an important attribute that people look at. And so, I do wonder if there’ll be more of a sort of power-law winner to the place where you want to originate an NFT. It may not be the case that all NFTs originate on this sort of canonical…

      Chris: But couldn’t you have trustless bridges across blockchains that preserve provenance?

      Jesse: Yeah. And I think that’s ultimately the solution. So, I believe in a world where literally every piece of media enters its existence as an NFT. Like, every photo you take on your iPhone, you know, every game asset is created as an NFT. And it probably doesn’t make sense to put all those very, very long tail of media assets on something like Ethereum, which is very expensive because it has a lot of security. Like, you probably put that on a side chain. But then I think as these assets take on social value and start to command more market value, they might migrate to the chain that offers the highest security, right? So, if you have a multimillion-dollar LeBron, you might not want that riding on some side chain, you might want that on Flow itself. And similarly, a photo that starts as inconsequential but becomes very important will maybe migrate to Ethereum for security.

      Chris: I guess the way I think of it is you’d have different blockchains with different tradeoffs. So, right now, Ethereum, of the non-Bitcoin programmable blockchains, is clearly the highest security blockchain. But you pay for it. You pay to do stuff per transaction for gas fees. So you could imagine a world where the actual activity is happening on Rally, or Flow, or something else. But then as it appreciates, you put it in the “vault” on Ethereum.

      I’d also say, I think that a lot of people will frame this as either or, Ethereum versus Flow. If you look at every computing resource in history — so, internet bandwidth, you know, PC, CPU power, just go through them all — demand outstripped supply by 10x. And, right now, you could imagine a world where tomorrow, you snap your fingers, Ethereum has sharding, proof of stake, all the good stuff, Optimism launches, you know, all the other Layer 2 launches. And then application developers would come up with some new clever stuff, and you’d very quickly be back up to not $10 gas fees, but $5 gas fees.

      I mean, look, you just pay all the above. You pay inherent overhead for a blockchain, right? The game theoretic consensus mechanism just makes it slower than in a traditional centralized system. I think the specifics, as you raised, Jesse, like, you know, how do you preserve provenance across chain is really interesting. I think there’s a lot of entrepreneurial opportunity for abstraction layer. So, like, a stripe for minting NFTs seems like a no brainer idea. Like, stripe for NFTs, whatever you want to…for minting and read write, and it abstracts away the underlying blockchain, you know, taking care of the metadata properly. There’s a whole bunch. We need more entrepreneurs because there’s so many great living fruit ideas.

      Jesse: Maybe we could talk about interesting intersections between NFTs and social tokens, So an author minted a blog post as an NFT on Mirror. And they also crowdfunded the creation of that blog post, which is sort of like a longform piece. And they said, “Look, if you want to see me write this investigative piece, back me to do it.” And the way that that crowdfunding happened was through crypto. And what the crowdfunders got was not just the piece, out there on the internet, but also ownership in the piece itself. Meaning, they were able to get fractional ownership of the NFT.

      And that was in the form of a fungible token called the Essay token. And what’s really cool is that Essay token then became a social token for this author, in that he started using it for gated access to a Discord. And he started layering on all kinds of other utility, where if you had this token, you could talk to him about his next piece, for example. So that’s one interesting example where the social token is sort of a derivative of the NFT. And then, as we talked about already, there’s all this sort of, like, additional utility programmed onto it. And I wonder if you’ve seen people doing stuff maybe the other way around or different configurations.

      Fractionalization of NFTs

      Kevin: Yeah. You know, I think it’s really funny. We start with fungible tokens, Bitcoin, Ethereum, etc., and then we create NFTs as a new building block. And then of course, then we take the non-fungible token and we fractionalize it and make the fractions fungible. It’s a funny world.

      I think we come up with some of these weirdnesses, especially here in the United States, because of the regulatory gray areas. And so I think, if you can sort of wave a magic wand and say, hey, like, there’s clear boundaries, and let’s just say we can talk about all the things we want to take about about, utility and so forth, without triggering any other things. I think a lot of people would start with fungible tokens. It’s just a lot easier to think about how do you create a community around something that’s more fungible and can be easily exchanged.

      Chris: I don’t totally agree, Kevin. I think it kind of depends on the community. I think there’s a lot of people. I think we all come from technical financial backgrounds. I just find people like that just kind of have a natural affinity for fungible tokens. I think a lot of the world isn’t like that. I think a lot of the NFT appeal is just that, it’s a picture. It’s a movie. It’s accessible.

      And it’s interesting, and it touches on culture and not just numbers.

      Kevin: I totally agree with you. I’m certainly not suggesting that fungible tokens are greater in any way than non-fungible tokens. I’m specifically talking about once you take a non-fungible token, and you fractionalize it into a fungible mechanism, I think once you start going into that …

      Chris: But the counterargument is Crypto Punks. That’s a great way to organize a community. And so it’s 10,000 punks, they look different. But it gives it this character. I think for a lot of people, that kind of metaphor is a better way to organize 10,000 people than 10,000, you know, indistinguishable, boring … it’s YouTube vs. Excel or something.

      Kevin: Each one of those are unique and different, right? And the characteristics and traits of them really matter. Those Crypto Punks are not fungible in my mind, right? Each of them are unique pieces of art.

      Chris: For sure, they’re not fungible, but they create a community of 10,000 people that feel an affinity for one another. You’re either in the punk community or you’re not. I was pushing back on your point that it’s because of regulatory constraints and things that people tilt towards NFTs. I think there’s a bunch of reasons.

      I think one of the really interesting things that goes on with social tokens is that they’re multiple uses. So, you could want a token for your favorite band to get backstage access. You could want it to be able to buy NFTs, and make donations, and participate in the economic life of the community. Or completely different motive, you could want it because you see yourself as a modern day A&R person who is going to predict the next band and make money off it. And it’s the fact that the very same token has those multiple purposes is very important.

      Because then you have the possibility for the A&R folks, the people that find these early bands, to come in, they kind of bid up the price. That, in turn, in your model, funds the actual musician to make music, right? Which in turn feeds the fandom. So the investment kind of activity, and the fan activity, and the creator activity, have this really nice kind of triangular feedback loop. And it’s the very fact that they’re the same thing as opposed to the old school model where you have sort of the investor comes in, gives money to the musician, buys up copyrights, then sells a different thing to the fans, right? The fact that there’s the same kind of… You see what I’m saying?

      Kevin: That’s right.

      Chris: To me, that’s a very big difference from the old world, and a very powerful difference.

      Digital tokens and music creators

      Kevin: Yeah, We’ve always talked about the internet as kind of this disintermediating sort of mechanism. And I think crypto does it even more at the economic layer of things. And I think what we’re starting to see, we’ll stay on musicians for a little while, like, starting with Chance The Rapper, is probably like the most prominent example of an artist that just wants to own all of his own rights. And more and more are realizing that they get their power from their fans. And that the more that they’re just wholly focused on serving their fans, the more successful they become. And this weird layer of rights ownership, and labels, and publishers, that then distort that versus just how you get your fans to support you to create the music that you love.

      Chris: And then when the fans have skin in the game, the fans become evangelists. And instead of doing all these old school things, advertisements, and other sorts of things, promotions, and all the things they would do in the old days to promote various kinds of creations, now the fans become the promoters, right?

      Kevin: That’s right.

      Chris: Because they have skin in the game. And that’s one of the remarkable things about crypto. Tokens, over a trillion dollars in value, lots of successful companies, you know, exchanges and such, and no one ever spends money on paid marketing because it’s all just done through kind of skin in the game peer to peer marketing, right? So, now musicians can tap into that, musicians, creators can tap into that energy.

      Jesse: Yeah, it means that fans become part of the creative process to a degree. Not only do they have a willingness to pay for the creative work, but they’re also essentially investors in the work itself. And artists or creators that lean into that will find a whole new way to create stuff. Because they can do it as a community. And not just a community that’s communicating with one another, but a community that’s actually pooling resources with one another to achieve things together. So, it sort of blurs the line between creator and audience. So I think that’s a big opportunity that we haven’t seen a whole lot of yet, but is coming.

      Kevin: Yeah, I think 3LAU is sort of experimenting with this, where you sort of sold the first track on his next album as something that would give you creative direction on that track as well as original ownership of that track.

      The best thing about working with creatives, whether it’s a musician, or a visual artist, or a gaming streamer or entertainer, is that, these are naturally creative people that once they get their heads wrapped around how a tool or a cryptosystem works, I think we’re just seeing the very, very tip of the iceberg in terms of the use cases because the technical challenges there will get solved, the friction will get removed more and more, there’ll be a lot of different approaches to this. But I think the most exciting thing is getting these tools into the hands of creatives, that then try all these new ways to create that alignment and community with their fans, and disintermediate the folks that haven’t been aligned with serving that community.

      I think the coolest thing that’s happening, is that we’re creating kind of an integrated way that creators can very simply create all different types of NFTs and denominate it in their social token. And, you know, we just think that those are such natural parings, right? So, you know, today, with just social tokens, we see our artists do things, like, you have to hold X number of our tokens to get access to the trove of music that’s here, or to get access to an AMA event or a virtual concert that we’re putting on, or virtual hangout that we see some of our artists are doing.

      And when NFTs come out, it’ll just be much more simple to say, okay, you hold X number of tokens, you are part of the fan club, but to get access to this event, here’s the NFT that represents that event access.

      And then so there’s a lot more granular things you can do. We’re starting to see artists experiment with things like creating physical representations and then digital representations, and then linking the two things together through their NFTs and social tokens. We’re working with an artist that sells sneakers and other physical merchandise. And then they’re creating NFTs of those designs and owning the NFT. They are now starting to say, well, if you own the NFT, you could trade that NFT in for the physical. We’re starting to see more experimentations with how does the physical and the digital sort of coexist.

      And the beauty of all of this is that when you denominate it in the social token, there’s so many other different economic activities that can happen.

      NFTs based on writing

      Chris: We’ve talked a lot about music and video games. What about other… Jesse, you mentioned writing.

      Jesse: Yeah. The writing one is interesting because it sort of illustrates how this expands in all directions and eventually will touch all creative services. So, if you think about how big ideas come into the world, very often they come into the world through blog posts or writing. Think about, Elon Musk’s secret master plan for Tesla, right? Like, that’s sort of a canonical blog post. And certainly, someone might want to own that blog post as sort of a representation of all the value that Tesla is creating in the world. And it’s an investment in Tesla’s success that’s separate from Tesla’s stock price, potentially, right?

      Chris: Owning that blog post is sort of like owning a kind of signed copy of the blog post, so to speak. It’s the cryptographically guaranteed one. Maybe kind of analogous, you know, if, I don’t know, Thomas Edison had a…I don’t know if they have it, but his original notebook or something. You know, that type of artifact.

      Jesse: Right, exactly. So the idea here is like owning Elon Musk’s secret master plan as an NFT, could be valuable. Jack’s first tweet as an NFT is valuable. It represents the sort of inception of his huge network. You can imagine, in the future, that the next Elon Musk or the next Jack brings their big idea into the world as a blog post that is an NFT. And, you know, supporters of them as founders, or people who want to see that idea happen in the world, crowdfund to buy that NFT from the author. And potentially, that crowdfunding is actually used to finance the creation of that big idea. So, you know, what starts out looking like, oh, people are just buying essays, could actually disrupt the way that new creative things enter the world and they’re financed.

      Chris: I guess what we don’t know is, will the economics work? I guess, one counter argument would be, such an exceptional blog post could be worth a lot, but the average stuff won’t be. I guess the counter-counter argument is the average person doesn’t usually need that much money. It’s sort of the Substack effect. If you take a writer with a million Twitter followers, who is getting paid X amount per year, and they go on Substack, and they get 1,000 people who are really excited, they can make 10x per year. Even if it’s not all some famous artifact, if it’s enough to fund the writing, that could just be enough to transform that industry and the creative activities there.

      Jesse: Yeah, and I think… I mean, maybe an analogy is, you know, startup funding, right? Like, startups don’t raise many millions of dollars in their first round. They raise a little bit, just enough to hire the team and get going, right?

      Chris: Sort of staged NFT sales.

      Jesse: Yeah, yeah, exactly. Like, I think you just need… I think, to your point, you just need a little bit to get going, right, and to raise money to make the creative work you want to see. And then from there, if you can build a bigger audience around that, you can sort of move up the ladder and raise subsequent rounds of funding. And hopefully, as you perform better in the market, you’re able to double down and reinvest to keep doing that.

      Chris: And Kevin, your model, that would be both NFTs and also a writer can just sell their coins as well, right?

      Kevin: We think of tokens and NFTs all operating together in a singular economy. This is just something that certainly comes from the videogame industry, where you would expect that you go into a video game, just using the Fortnite example of, get the V-Bucks, the V-Bucks allow you to buy all sorts of different things in the game. And when we talk about things like backstage passes, or that essay, or something else, those things are best as an NFT.

      But then what do you do to transact with that NFT? What if you, as an artist, let’s say your first creative work sells for $10,000. Let’s say at that point, you create your social token, and you denominate your NFTs in your social token. Well, now you have a way for people to say, okay, great, that creative’s first work was worth $10,000. Their second work, the audience valued it at $20,000, and their third work was $30,000.

      The social tokens should then capture what the market thinks about the sort of total value of that economic output will look like over time. And so I think there’s really interesting economic forces between how you create your NFTs and what does that represent? And what does your social tokens represent? And how do they all work together in a singular sort of economy that you, as the creator, you control, you own 100%. And I think that’s a really powerful way to both give all forms of fans, and community members, and crypto members, kind of a way to participate in these economies. You may want to, you know, buy that NFT, because you’re a true fan and you just love it. You may want to buy that NFT because you’re an A&R and you want to speculate on what these things could potentially be worth in the future. Or you could just participate in the social token in all sorts of different ways. I think we’re going to be on the forefront of experimenting with how these things are intertwined, and all put into the control of the hands of a creative.

      The role of big tech platforms

      Chris: What do you guys think the role of the large tech platforms will be in this world?

      Kevin: Well, I’m pretty passionate about this. I’ve tried to build businesses in the past, certainly that have been at the mercy of some of these big tech platforms. I look at what Epic is doing right now, and I know, Jesse, you were at Spotify for a while. There are very public emerging battles between big tech and some of these traditionally more application-focused developers. And I love what this does to the world.

      As I was thinking about building a new company or building a new project, and thinking about building that on Ethereum was so liberating. Because I’ve been building on Apple, Google, Facebook platforms for a decade-plus, as a game developer. And there’s a ton of benefits and value that comes with it, but there’s a lot of headache too that comes with building on these other platforms, where the policies are changing all the time, the fees are changing, the rules are unclear. Maybe they the platform ends up competing with you in the case of music or some other categories. So it’s tough.

      This is why I’m so in love with crypto as a builder because building on Ethereum, I don’t need to worry that Ethereum is going to try to go public someday, they’re going to change the way that the rules work or the fee structure works, so that they can meet their numbers for the next quarter or whatever it is. There’s not even a company, there’s not a CEO that runs it. The idea that this thing is a permissionless blockchain that anybody can build on top of was such a game changer for me as a builder.

      And I think our approach to Rally was to do a lot of hard work so that we can make the same promises and commitments to the creatives that we work with. If we work with you to help you build your business and represent your brand, your fan audience, your community, through tokens, both fungible and non-fungible, you own this. You set the rules. You know, we do things at the protocol level to ensure that all people can participate equally and fairly with transparent rules, but we want to make sure that you, as the creative, you truly own this thing.

      Jesse: Yeah, what’s happening in crypto definitely flies in the face of the way big tech platforms work right now. I think another lens to look at it from is just a complete inversion of their business model. And that’s because, like, you know, traditional big social media platforms, they own all the content that users post on it. And that’s because somewhere along the way, in the terms of service, users agreed to upload content to the platform for the platform to monetize it as they see fit. So, to be clear, I’m not talking about traditional copyright, like, the creator still retains that. But you are transferring some rights to the platform to monetize content that you upload, however they want to do it.

      And with NFTs and with social tokens, the amazing new thing enabled by web3 and crypto, is that creators can just monetize directly. When you create an NFT, you’re sort of like uploading your work to the blockchain directly, and then developers can build on top of that content. In other words, the blockchain becomes this sort of universal library of media that any developer can build a social feed on top of or content feed on top of. But the creator retains ownership of their work and thus can monetize it without a third party taking a large cut. And so, these platforms traditionally have relied on being able to monetize creators’ work on their terms, and now creators get to set the terms and monetize directly.

      Kevin: And if you start with that kernel of the creators create the content and then developers build interesting metadata and usage around that content, that then becomes the social graph itself.

      So think about how, for big tech platforms that rely on marketing and advertising, yes, they create a simple platform for users to share, and create content, and build an audience and following, but it’s really a lot of the metadata around that content, who’s following you, who’s liked what in the past? I think what’s going to invert now, is, once you start with the content being on chain, who owns that content? Who’s owned it in the past? What are all the other metadata that’s associated with that? If you can see all of that, and that exists at the public blockchain layer, you then take it away from being this treasure trove that an advertising-based company can uniquely have as their advantage. And you open it up to the whole world anybody can look at that data. Anybody can look at that social graph and interest data, and then figure out how to build unique new applications and services on it.

      Jesse: Yeah, right now, on social media, everything is sort of in 2D, right? Like, you have an image and it’s just a rectangle on your screen. And then you have some metadata associated with it. But if I copy-paste that image and put it somewhere else, it loses all its connection to the creator, its history, what it’s about. And now all that metadata can live on chain, any developer can access it. So, as a result, you know, this image goes from being a two dimensional box, to taking on some Z-access, where you can peruse through its entire history online, and put all the context on display in new areas where that image is shared.

      Through that same channel that information is being surfaced, value can also flow. There’s this cool thing you can do with NFTs where you can impose royalties that flow back to the creator every time the asset changes hands. So that’s an example of, through the same channel, that information on who owned this thing in the past, well, value can also flow through that same channel.

      Chris: Awesome. Thank you, both Kevin and Jesse. Great talking to you.

      Kevin: Thanks for having us.

      Jesse: Yeah, thanks.

      • Kevin Chou

      • Jesse Walden is the founder and managing partner of Variant Fund, investing in crypto networks and founders. Previously, he was a partner at a16z Crypto and cofounder of Mediachain (acquired by Spotify).

      • Chris Dixon is a general partner at a16z, where he leads the crypto/ web3 funds. Previously, Chris was cofounder & CEO of startups SiteAdvisor and Hunch (acquired by eBay); and an early blogger at cdixon.org.

      All about NFTs

      Jesse Walden, Linda Xie, and Sonal Chokshi

      This episode is all about NFTs. It seems like nothing has caught on and spread into mainstream interest like NFTs, where one hears everything from “I’ve never seen anything like this before” to “is this like ICOs all over again” to “it’s just a jpg I don’t get it” to “but what about the energy use!”

      So, in this special deep-dive episode from the a16z Podcast network, we break down everything you need or want to know about NFTs — while cutting through the noise for what’s hype/ what’s real, as well as where are on the long arc (and sometimes seemingly sudden tipping point!) of innovation (apparently, Google trends data showed that interest in NFTs recently surpassed interest in cryptocurrency). Editor in chief Sonal Chokshi interviews friends of a16z crypto Linda Xie, co-founder of Scalar Capital and former Product Manager at Coinbase; and Jesse Walden, founder at Variant Fund and former co-founder of Mediachain Labs (which was acquired by Spotify, where he was then an R&D lead).

      This episode is posted on both the a16z Podcast show and 16 Minutes as one of our “2-3x explainer episodes” of topics that keep coming up over and over again in the news (past such episodes have covered everything from Section 230 and Tiktok to GPT-3 and the opioid crisis).

      Show Notes

      • What NFTs (non-fungible tokens) are — as well as the properties of crypto that enable them, just to set some big-picture context [2:07]
      • What forms they take, and what is and ISN’T an NFT — including where “social tokens” and the creator economy do and don’t come in [11:18]
      • Common myths and misconceptions — from ‘just a jpg’ to the frequent question of energy use & NFTs [18:23]
      • How they work — as well as the broader ecosystem around NFTs [29:35]
      • Discussion of different ecosystem players [35:08], including DAOs [44:02]
      • Various applications, now and next — touching briefly on how to think about NFTs, whether you’re an artist/creator, developer, or institution [49:32]


      Sonal: Hi, everyone. Welcome to the a16z Podcast network. I’m Sonal, and this episode is ALL about NFTs. And, as with our other special deep-dives, we cover everything you need or want to know about NFTs, while cutting through for what’s hype, what’s real, as well as where are on the long arc (and sometimes seemingly sudden tipping point!) of innovation.

      We start for the first 10 minutes by discussing what NFTs are and how crypto enables them, just to set some big-picture context. The next 10 minutes, we cover what forms they take and what is and ISN’T an NFT — including where “social tokens” and the creator economy do and don’t come in. Then for the next 10 minutes we cover common myths and misconceptions, from “just a jpg” to later going into the frequent question around energy use and NFTs. But about 30 mins in, we quickly share how they work, as well as the different players in the ecosystem.

      Throughout, we of course cover various applications, now and next. And finally we touch briefly on how to think about NFTs. Whether you’re an artist/creator, developer, or institution, this episode is for everyone.

      And, as with past such 2-3x explainer episodes (as I call them), it’s being posted on both the a16z Podcast and 16 Minutes feed, our show where we talk about tech trends in the news, including topics that keep coming up over again [ICYMI: past episodes include explaining Section 230; Tiktok; GPT-3; and the opioid crisis – you can find all of those at a16z.com/16Minutes. We also covered the historic auction at Christies this month, where an NFT by the artist Beeple sold for $69 million; and I mention that only since we reference that event in this episode.]

      Finally, since our podcasts bring you insights directly from the experts, the guests I’m interviewing today are two close friends of a16z crypto: Linda Xie, co-founder of Scalar Capital and former Product Manager at Coinbase; and Jesse Walden, founder at Variant Fund and former co-founder of Mediachain Labs, which was acquired by Spotify, where he was then an R&D lead.

      To be clear: NONE of the following should be taken as investment advice. For more important information please see a16z.com/disclosures.

      Terminology & foundations

      Sonal: So. Let’s just start with a quick set of definitions: What IS an NFT?

      Linda: So, NFT stands for “non-fungible token”, which is just a term used to describe a unique digital asset, whose ownership is tracked on a blockchain.

      This can be a really broad set of assets from: digital goods, like virtual lands and artwork; to a claim on physical assets, like real estate or clothing items.

      Sonal: What I heard you say there is not just digital, because it *can* cover something physical as well, that you can essentially represent as NFTs.

      Linda: Yah. It’s a really really broad space; it’s exciting to see NFT art really take off, but this covers a lot of different industries as well.

      Jesse: So, I like to focus on the digital side of things a little more, and, a metaphor that I would offer as a definition is NFTs are a way to make digital files ownable — instead of a financial asset, you can now own a digital media asset on the internet.

      And that’s why the file metaphor is apt: You can now own a JPEG, own an MP3. And, what you’re essentially doing when you create an NFT, it’s sort of like metaphorically ‘uploading’ that file to the blockchain — such that anyone can track its provenance and attribution.

      Sonal: So, Dixon described this in a recent blog post, very simply put, as: “NFTs are blockchain-based records, that uniquely represent pieces of media” — or in your words, Jesse, a file.

      One more word to focus on is the “fungible” in the non-fungible token, which is that you can represent these items uniquely — I just want to really emphasize ‘cause, again when you think of $1, that’s fungible (or even a single bitcoin arguably is fungible) — but something fungible is interchangeable, replaceable; it doesn’t matter what dollar I have, as long as I have a $1.

      But in this case, something is “non-fungible” means it’s super unique; and we can go into like what that means in a moment — but before we do, let’s talk now about the underlying crypto aspect of NFTs… not the specific crypto protocols, but maybe more broadly, what are the properties of crypto — ‘cause we don’t wanna make this conversation about crypto per se, but about how crypto enables NFTs.

      When you think about the physical world: sometimes it involves a notary; like a third-party bank; it involves someone to (in the art world), like provenance-tracking through certificates… — this ability to own and track a digital file, without a third-party player intermediary, is key.

      Jesse: That’s right, you depend on the bank to maintain the ledger; or the title to a property that you buy, there’s some property registry that the state or the city maintains. So you’re always dependent on a third party to track the attribution of ownership: how the title changes hands, how bank statements get updated.

      And bitcoin changed the game because it enabled this public, decentralized ledger — where no one party is in control — and yet each individual owner of a bitcoin is able to verify their ownership using cryptography. As a result, you don’t have to depend on a single third party to verify ownership.

      Linda: Yeah, middlemen are tracking ownership for people of all these different assets — and they’re taking fees for the service; they’re preventing some people from using the platform — and, what’s really powerful with crypto is you have all these open protocols that you can kind of plug into each other.

      And so, when you have NFTs, you can plug them into decentralized systems and be able to trade these NFTs with anyone in the world, and have that be instantaneous. You can also imagine plugging into using your NFT as collateral; so let’s say you have video game items that’re worth a lot of money, you can actually imagine taking a loan out from them.

      And so NFTs on the blockchain allow anyone to permissionlessly own, issue, trade them.

      Sonal: And the other property of being able to track provenance; which has essentially a built-in secondary market to it — which is this idea that not only do you track the provenance, but you can actually track the financial benefits that accrue as a result of that built-in secondary market. This is particularly true in cases of digital artworks, etc.

      Jesse: Yah, the secondary *resale* of an asset can be programmatically constructed, such that anytime the NFT changes hands, a portion of the resale value goes back to the original creator.

      Sonal: And by when you say “programmatically”, automatically, that is a distinct property of crypto — specifically, smart contracts that you can do that type of programmability of a contract.

      Jesse: Yeah, it can be totally automated, totally transparent — Contrast that with royalties in the music industry, which is like a completely opaque system with many layers of middlemen that are each taking a cut, right.

      It’s a wildly more efficient architecture; that’s uniquely made possible by blockchains and smart contracts. The blockchain’s really good at tracking the history of things — Sonal, if you send me one bitcoin, everyone can see that you have one less and I have one more <Sonal: right> — and the history of that transaction is forever sort of enshrined on the blockchain. The same is true of non-fungible tokens in that, when they’re incepted or “minted”, they’re signed by the creator using their cryptographic keys — which now enables anyone to see okay this file was signed by this creator or this person — that message is constructed in the same way any other cryptocurrency transaction is constructed.

      Thereafter, that NFT lives on the blockchain alongside all other transactions, and everyone can see it. And so if that NFT changes hands, and say Linda buys my NFT, everyone can see I transferred ownership to Linda. And as a result, we start to build this very rich history of the interactions people have with media on the internet. Whereas today, think about an image you see on Instagram: You could screenshot that; you know, crop it; and then paste it on another platform, say like repaste it on Facebook. And as soon as you do that, you break its entire history, its entire provenance — you no longer know who made it, what it’s about, where it originated… And, with this new sort of architecture, we can now sort of have a z-access into the entire history of any piece of media on the internet.

      And: through that channel that information flows, value can also flow.

      Sonal: Concretely for artists, this means an artist today who may have created a work 20 years ago — and that work completely appreciates in value, but the last owner is the only one who benefits from that — if the programmatic arrangement is that that artist continues to get value, they can always get paid on this built-in secondary market. Like, if it later becomes millions of dollars versus $500 for a painting, then you’re getting money back each time it is sold. Which is not possible before.

      Jesse: And I think, important to note, that’s just *one* of the possible arrangements that can exist, when the rules around monetizing creativity can be expressed as code by any developer and any creator on the internet. 

      Linda: The ownership history is really important — the ownership history is something that is really uniquely accessible on the blockchain, because everyone can see, and therefore some items might be more meaningful to certain people. Let’s say “Magic: The Gathering” has a tournament where this deck of cards actually won this tournament, you might want to buy these set of cards because they’re historical, and the winner of these games actually used this deck to play.

      From the art perspective, just imagine your favorite musician or creator owning a piece of art. And, now that ownership is just tracked in the blockchain, that piece of art might become more valuable to you because of who’s owned it in the past.

      We also have a lot of projects that are working on fractionalizing NFT art: So, splitting up these NFTs into multiple pieces; and these individual pieces are also tracked on the blockchain, and you can trade them through decentralized exchanges.

      So, it’s really powerful when you can plug these NFTs into all these different crypto protocols, because in a traditional system, these middlemen aren’t plugging themselves into all these other companies and middlemen. You can kind of freely do whatever you want with these NFTs, which I think is a really big difference.

      Jesse: Yah, I think it’s important to contrast the way NFTs work to the way the traditional web works; so, with social media today, when you share a file or share a piece of media, you upload the file to the platform. And what’s actually happening under the hood is you’re “copy-pasting ownership” of the file to the platform: What I mean is that somewhere along the way you signed the terms of service that allows for the platform to monetize that piece of content as they see fit, and maybe they give you a cut of the revenue, maybe not — but the platform gets to make that call. And, they also get to make the call on how that content is consumed, and there’s not a whole lot of innovation going on there because any developer who plugs in to try to innovate has been shut down in the past.

      Now, contrast that to NFTs — and I’m going to run with this metaphor of uploading a file to the blockchain —

      Sonal: Keep goin’, I love someone who owns a metaphor… <chuckles> <Jesse chuckles> do it! <laughs>

      Jesse: Okay, so, with this metaphor if you’re uploading files to the blockchain, and then those files become NFTs and they behave in the way that other crypto assets behave, that means that they’re permissionlessly accessible to anyone, anywhere with an internet connection. The implications are that any third-party developer can then innovate on the way that media is consumed: Like how the audience sees it, how people can interact with it or program it.

      So, one way to think about what’s happening with NFTs is we’re building this universal,  open media library — on top of which any developer can build the next Spotify, or build the next Instagram, or build the next Facebook — and when there’s a lot more competition, there’s gonna be a lot of benefit to consumers… and likely to creators as well — because as Linda mentioned, all of this can happen without the traditional middlemen taking a cut of the value that’s flowing between the creators and consumers.

      Sonal: That’s great.

      What an NFT is and isn’t

      Let’s get a little specific, though — let’s actually talk about the forms NFTs are taking, specifically. I think this is a great place to help tease apart what’s hype/ what’s real, as is the premise of the show. And so far, I’m actually having a hard time — and I’m someone who’s been covering this space: I mean bitcoin since very early on, Ethereum since very early on, NFTs since very early on — and I am honestly confused myself! So, maybe you guys can just help break it down.

      Just to quickly recap, Jesse, you’re saying any media file; Linda’s saying any good, digital or physical — that leaves pretty much anything… So specific examples include things like:

      • art;
      • it can be in games;
      • in music — there are audio NFTs (this has been really interesting to me lately);
      • there are blog posts, like I see people on our friend Denis’s site talking about making NFTs of blog posts;
      • Brian Flynn wrote about token-gated newsletters
      • And another interesting example recently (it was a self-proclaimed first, but likely true): Someone wrote about how they created the “first ever tokenized crowd-funded equity research report”.

      …Can you guys just quickly help tease apart what *is* and *isn’t* an NFT? It seems like everything is!

      Linda: I really think that anything <chuckles> in the world can be an NFT… as in, anyone creates something that is unique, that can be owned.

      The problem with physical goods is that you do have to have someone custody it, so there is a process behind that of having to make sure that you can audit that in the real world: And maybe multiple people own it, and it can’t be moved just by one person; so, there are pieces to that — but otherwise I find it to be an extremely broad category.

      We’ve seen really cool things come about, where we’ve had the token-gated newsletters, which you mentioned — basically needing to have a certain number of tokens in order to access this newsletter. And people are doing like token-commissioned permission chats where these tokens are required to enter the chatroom and start talking so, you know everyone has like a level of skin on the game. You have to own like a certain amount of tokens in order to enter these groups: it proves that you have some sort of ownership into this community that can be adjusted over time. (The idea is that even one day that there could be DAOs formed where token holders can vote on how many tokens are required to enter this newsletter or chat group or something.)

      And that piece, it’s kind of tangential to NFTs — I don’t necessarily think that social tokens themselves are NFTs, because sometimes people are creating these tokens that they’ve minted like a million of them; but if the creator of this group is issuing individual badges or unique items within that, then that can be an NFT.

      Sonal: Can you say more on what you mean by “social tokens”?

      Linda: Yeah; social tokens are just a really broad category of tokens that are issued by individuals or communities. So sometimes it’s- other terms are used like personal token, community token, creator token — but social token’s kind of the term that encompasses all of it.

      And there’s just a bunch of different experiments happening in the space: So we’ve seen people tokenize their time; so one of these tokens equals one hour of their time, and that becomes freely traded. We’ve also seen someone like R.A.C (he’s a Grammy Award-winning recording artist) tokenize his social token, and his token holders get access to this private Discord group, and then they receive like all these additional benefits… and he retroactively distributed to his supporters — so this is a way that creators can interact with, and reward, their early supporters.

      So it’s a really broad category and it can really be anything associated with an individual or community.

      Sonal: And how is social tokens similar and adjacent to NFTs — and then when is it *not* NFTs? Can you help distinguish there, just to help the understanding of what is / isn’t an NFT.

      Linda: Yah, sometimes social tokens can be NFTs in that a creator is issuing some unique piece of artwork directly to their fans — but in a lot of cases, social tokens can also just be fungible tokens. So like R.A.C. token, they’re all fungible so you can basically hold like a certain level of them, and that can always be traded and bought back and it doesn’t really matter which R.A.C. token you’re purchasing. So those are kind of more adjacent.

      And I think the reason why social tokens and NFTs get lumped together a lot of times is just because it enables the creator economy, it enables creators to engage with their fans directly… and so, these are often tied pretty closely together.

      Sonal: Right, but basically the bottom-line is — if it is fungible, it’s not an NFT; and if of course if it’s non-fungible (hence non-fungible token), then it is an NFT.

      Linda: Yah. 

      Jesse: I think the line between fungible and non-fungible tokens is blurry for a reason, and that’s because the interplay between the two is enormous:

      • You can take a non-fungible token, and turn it into fungible tokens by fractionalizing it;
      • And then you can make those fungible tokens — which represent a piece of the original NFT — you can make those into a social token.

      So there’s this token called B.20 which is a fungible token. And it represents a claim on some of Beeple’s NFTs, which an investor bought and essentially fractionalized ownership to. And now that that B.20 token exists, it can be programmed into all sorts of other value: So in addition to owning a piece of a Beeple, you can imagine some third party spinning up say, a Discord server and saying you need a B.20 token to come in here and hang out. That’s an example where it started with a non-fungible token; Beeple created it; then a collector bought Beeple’s non-fungible token; fractionalized it into a fungible ERC-20 token (that’s the B.20 token) — and third-party developers can remix and add new experiences.

      Linda touched on all of this, but the interplay between the two is important to note that you can easily take a non-fungible token and fractionalize it into fungible tokens, that then can become these social or community tokens. And so that’s a fun sort of design space to explore.

      Sonal: I mean we’re talking so far about kind of digital versions of what already happens in the real world, being able to do things in different ways. But if you think about what’s NOT possible right now in the real world, the idea of fractional ownership is super important and goes to the things that crypto can uniquely do. ‘Cause right now, if you want to fractionally own an artwork — I mean, I guess there are some very kluge-y websites where you could go and kind of combine resources with somebody — but guess what <chuckles> you can’t really split a physical artwork! So, it kind of reminds me of that story, I don’t know, it’s like a Bible story about like splitting the baby; like, who’s the real mother? But the point is, you can enable fractional ownership.

      I still am stunned at this idea with Top Shot, that you can essentially buy and own a *moment*, a favorite hit moment in a physical sports game. And by the way Linda, I’ve been meaning to text you about this, but I’ve been addicted to watching CLOY (Crash Landing on You, the K-drama); honestly when I think of all these amazing moments in a show like that, I know this sounds nutty, but what if people could bid and own moments on their favorite TV shows — as if it’s theirs — even if millions of people can watch it! Like, the idea that you can own it, and it’s part of your identity, is so freaking awesome.

      Linda: That’s what I love about crypto, you can have these concepts that you never really would have thought about otherwise — about being to own moments, and media, and people’s lives — imagine like YouTube creators streaming what’s happening to their life, and be able to own really exciting moments of it. I mean there’s so much possibilities there… it’s really exciting to see people become so creative with NFTs and what you can own.

      Common questions

      Sonal: On that note, let’s actually switch to what’s overblown or not, and tackle some common myths and misconceptions — everything from energy to whether it’s a hype cycle. And of course we’ll keep talking about the applications, but let’s first dig more deeply into what it means to own something digital.

      Start with this common phrase: “It’s just a JPEG”. How would you guys address that comment — “like ohmygod someone spent f’ing $69 million for just a JPEG?!” I mean, Jesse, you’re saying it’s a file. Guess what? On the internet, files are pretty worthless, and infinitely replicable.

      Jesse: So I would say that the number of times a file has been reproduced on the internet, is directly correlated to the value of that file’s NFT — meaning the more times a piece of media gets shared online, the more social value it has.

      To make this concrete, it’s helpful to think about a very well-known piece of traditional art, like the Mona Lisa. Where the Mona Lisa has been reproduced probably a zillion times — it’s on every t-shirt, postcard sold at the Louvre; you can see it anywhere on the internet — yet there’s ONE Mona Lisa in the Louvre Museum, and ownership of that piece of artwork is incredibly coveted, incredibly valuable.

      Sonal: I have to admit I’m getting a little tired of the Mona Lisa analogy, but it is a useful one <chuckles> — the key point and what I really heard you say, which is so counterintuitive, is that something is more valuable the more that it is replicated…

      And in fact, it just makes me think of how in general, as the world of the web became more about abundance versus scarcity — you know, the long tail is that you can have infinite choice on the web — it is really fascinating how people have been, when they went past the limits of the Blockbuster shelves in the physical-goods world, it’s interesting that Netflix would do things like binge seasons and drops, that kind of created this digital scarcity — like a limited-edition effect; like this thing is going to be on for three months, and then it disappears.

      So it’s another analogy of this interesting relationship between something not being necessarily rare or scarce — and in fact, even infinitely replicable in the case of files and JPEGs — but you can enforce this digital owning, or even a piece of it (if you want to go into the fractional ownership bit), and that is incredibly unique.

      Jesse: That’s the idea — is you’re not owning to try to be the ONLY person who can access a given JPEG or piece of media — rather you want to own the piece of media that everyone else sees.

      Memes, you know, travel the internet at a rapid clip; they get shared infinitely — you can now own a piece of internet history, or the most viral meme. And I think very soon we’ll see that these owners are credited, socially, on platforms where that work is distributed.

      Linda: We’re seeing that play out with CryptoPunks, which is just one of the earliest NFT projects. And we saw two CryptoPunks sold for $7.5 million each — one of the sellers being Dylan Field, who’s the CEO and co-founder Figma — and what’s really interesting about this is, yes, anyone could just copy this image…

      Sonal: …He, himself — sorry to interrupt you — he, himself, could copy his image, because he had it as his profile photo for a while, and he had to like (he didn’t have to, but he chose to) take it off his Twitter profile. Which I thought was so funny. But keep going.

      Linda: Yeah, exactly. So anyone can copy this; but if you actually look at the CryptoPunk NFT itself, you’ll see who owned this.

      And the fact that somebody had owned this so early on… it’s almost become this status symbol where people want to demonstrate that they can prove their ownership really early on in this space. And so that ownership history also really matters: Being able to discover an artist or creator really early on, and being like one of the first supporters, and having that tracked on the blockchain. Future times when that is sold, you can prove that you were this early adopter, and that is very valuable to people.

      Sonal: There’s a great analogy I heard, to extend what you said even further — but basically, if you think of the placard next to an artwork like at MoMA or some famous museum — it tells a description of what the art is, the materials, the date; the artist, if it’s not someone anonymous — and then it says at the very bottom, you know, sometimes in very small letters, who owns it, or it’s lent by someone on this collection so-and-so. And not everyone pays attention to that, because most people just care about the art, and then some people actually care about who owns it.

      But now, you’re doing that exact same thing but on the internet, where the whole world can see it — not just like going physically to MoMA and seeing it. Or even to use Jesse’s analogy, Mona Lisa and seeing it in the Louvre; it’s like not only can you have that kind of insider notion of the ownership, but you can make it more outsider facing by letting everyone see it. That is pretty powerful.

      Jesse: Yah, and developers are gonna lean into that. Because all that data about who owns it, where it came from, what its history online is — that’s just an API call away. And, right now, again it’s very difficult for developers to find all that information on web-2 platforms because the history of media is not tracked architecturally in the way that blockchains track it.

      So it’s like a 100 times easier for developers to surface that information — and that’s why I think you’ll see that little placard in the museum — you’ll see the digital equivalent of that on all social surfaces in the near future.

      Sonal: Did you guys see Matt Levine on Twitter, he said this line that “NFTs are a new form of tradable ostentation rather than a new form of tradable ownership.” Did you guys have any reactions to that?

      Jesse: I mean ostentation is one of the reasons people might collect NFTs… but it’s not just the speculative value, or being ostentatious about being the owner — increasingly over time, I think owners of NFTs will start to realize more and more sort of compounding utility as developers build new spaces for you to bring your digital property. For example today, you can buy a piece of digital art as an NFT, and you can bring it into a virtual world (like Cryptovoxels or Decentraland) and display it there. And that’s a very early example of a third-party developer who has nothing to do with the creator of that NFT, being able to build a new virtual experience — that’s just the tip of the iceberg; there’s, you know any third-party developer can then build on top of Cryptovoxels because it, too, is open-source and permissionless.

      And so what you start to see is this sort of Lego-block approach to building new experiences, where developers can build more with less, and the innovation compounds much more quickly. And so, that statement undervalues the possible utilitarian nature of being an owner.

      Linda: Yah, I don’t really agree with that statement, ‘cause I think it dismisses a lot of the use cases. And if we even talk about the more traditional stuff — like NFTs representing tickets or financial assets or real estate — these are just more efficient ways of transferring, and without having as many paperwork and middlemen involved.

      And so this is a net benefit to society, versus people trying to display their wealth.

      Sonal: I’m so glad I asked you guys because again the premise of this show IS to tease apart what hype/ what’s real, and it is interesting that someone whose work I deeply respect… has an interesting observation — and to hear you guys push back on that.

      What do you make of the comparisons that people are making to the ICO boom, what would your response be to that? ICO boom being “initial coin offerings” — playing off the term for IPO, obviously initial public offering — but in that case, it was more risky people argued, because an ICO was before the thing even existed. Like at least in an IPO, the company exists. By the way: Our friend Nick Tomaino made a simple observation that an NFT is a concrete product, a digital good, not a promise about the future, which I thought was a good argument. <Jesse: Yeah. I like that.> Any thoughts on what the hype/ what’s real on, “oh no, we’re in another ICO boom; it’s like tokens all over again”?

      Linda: It just reminds me a lot of 2017 when a lot of people came into this space — and the word ICO was thrown out a lot, and there was definitely a lot of hype around that.

      But, through that process, a lot of really incredible projects came out of crypto. And, a lot of people who joined the space ended up staying because they saw that it was a lot more than just-get-rich quick; and there were communities, and really unique ways of creating value in this world.

      So, yes, there are a lot of people that have come into this space, wanna just make some money off of it — but there are a LOT of really creative people that have joined the space and are going to stick around, and experiment with what’s happening, and really build some things that we have just never seen before.

      And it is nice that this time, it’s artists who have just worked so hard and are getting rewarded for this type of work. Someone like Beeple, who has been working on this craft for so many years, this is something that people are valuing.

      Sonal: I mean, on Beeple specifically his work was digital from the beginning anyway, but he’s essentially creating — and this goes back to the definition, this kind of one-of-a-kindness — because it is an NFT, it is unique and trackable that way.

      I do love that… but there’s no question there is like a hype cycle going on, we’re at that moment in whatever the Gartner curve, or whatever the framework you wanna use; there’s always a trough of disillusionment phase… I guess I just need a little bit more to understand, okay yes, this is very exciting — but right now, in this moment in time, how do we assess that it’s working / or not working?

      Jesse: I think the question you’re getting at is what is valuable, which NFTs are valuable?

      And the answer to that question is kind of like answering the question, “what is art?”; and my answer to that question would be whatever the beholder thinks is art.  And similarly, whatever you know, the market thinks is valuable… is a valuable NFT. And that’s why I think you’re seeing such wide-ranging experiments in what can be transacted as NFTs, from blog posts, you know to digital art.

      And there are a lot of niche groups that want to own you know an item that’s culturally relevant to them, for various reasons — whether it’s for speculation, the idea that they might be able to resell it in the future; or, you know, because they just want you know their name on this sort of digital placard next to the item to say “I supported this creator”, right like I wanted to you know support their work.

      So there are a lot of different reasons people might value NFTs, and there’s a lot of different subcultures and value systems that you know comprise this market, and that’s why it’s sort of expanding in all directions.

      Sonal: So the buzz is not necessarily a bad thing.

      One of the things that came up a lot in the early days of the history of NFTs, starting with CryptoKitties, was the scaling problem, and the fact that Ethereum was not ready for that level of excitement yet — and it pushed a lot of solutions into thinking about scaling. So some of the hype cycle in 2017 actually led to good infrastructure improvements, and the installation of things we needed…

      I mean it definitely makes me think that Mediachain was just a bit too early, actually. I remember… even before you guys founded Mediachain, always having this problem in the creator world of having to track libraries of digital assets, it was very difficult to find out — ‘cause the information was not coupled with the JPEG itself — forget even who owns it and who to pay, like you don’t even know who made it. And this is true of memes, everything, on the internet — which you know IS about remix culture, and extensibility, and-and composing things and combining them.

      Jesse: Yeah, a lot of the ideas that are being realized around NFTs are ideas we were exploring back then. And there were two critical things missing from the ecosystem at that time: One was the ability to easily create a token — that’s uniquely enabled by a smart contract, and smart contract platforms like Ethereum, and Flow, and others. The other thing that was missing was markets for these digital assets: So, we now live in a world where roughly 10% of Americans own cryptocurrency. And so the idea that digital assets have value is sort of a prerequisite for digital MEDIA assets, like NFTs, having value.

      To Linda’s point earlier, the 2017 market was a prereq for the NFT market today. So, the markets had to come first; markets drive — you know, they’re volatile, and they drive these speculative frenzies — but they also drive infrastructure, and mental models that stick.

      How NFTs work

      Sonal: That’s a perfect segue to the next thing I wanted to talk to you guys about, which is the broader taxonomy of the players and the ecosystem that’s already emerging around all this.

      And before we do, let’s quickly talk about how they work, to help make it concrete — like step 1-2-3 to minting an NFT, trading it, doing whatever?

      Linda: Okay; step one would just be deciding to put your work as a representation on the blockchain. And so “minting” involves really interacting with the smart contract, and submitting that — there are different marketplaces that try to make that really easy for you to do it. And so some of them will have a button that you’ll click to mint this process, you can select different attributes of like what’s the name of this piece of artwork, what’s kind of the royalties involved if there are secondary sales, like how much do you want involved?

      So, a lot of these will make it super easy for you to go through that process. I think the hardest part actually, is getting set up with a wallet, and onboarding yourself into accepting cryptocurrency and that piece — but there are marketplaces now that make it accessible.

      And some marketplaces will also maybe have you go through some onboarding process — and so they might have some due diligence on the artist; and making sure this is um real artwork and not copied by some other artist; and making sure it’s really high-quality pieces — and there are others that are just like hey, anyone can mint this.

      So, it’s quite a spectrum right now.

      Sonal: That’s great… but, I gotta set up my MetaMask, and like what does that even mean? Can you guys explain the wallet part too as well, ‘cause one theoretically does not necessarily have to actually interact with cryptocurrencies directly — so, if you guys could break down really quickly that bit too.

      Jesse: Sure I can take a stab at that. So, the concept of a crypto wallet boils down to what’s called a public and private key pair — So, basically you have a public address on the blockchain, which is where your assets/ your stuff is associated: So, Sonal has a public address, and you can tweet that out and say, “Hey, I’m Sonal, here’s my public address, and here’s all my stuff.” And your Bitcoin can be at that address; on Ethereum you’d have a different address, and all your stuff on the Ethereum blockchain would be associated with your public key.

      And then there’s the private key — the private key is what unlocks the transfer of assets in your wallet. So you need the private key to unlock stuff at your public address, and that’s the concept of a crypto wallet. MetaMask is the most popular wallet for Ethereum, and it’s a browser extension (you can install it on any popular browser); and, essentially what it’s doing is it’s setting you up with a public address for the Ethereum blockchain, and a private address.

      And what’s critical to note: Is that cryptographic key pair, it doesn’t belong to MetaMask, it belongs to YOU; MetaMask never sees the key pair, they don’t have any of the information, it’s yours. And that comes with a lot of responsibility, because if you lose your private key, you lose all your stuff — it’s kind of like cash in your wallet. And that’s why it’s called a “wallet” (even though it’s a little bit of a misnomer, because you can only do so many things with your physical wallet) but I think the reason that name has stuck, is that your cryptographic wallet behaves like a physical wallet in that if you lose it… all your cash is gone.

      Sonal: Okay, so now continuing the process, so we understand now how the wallets work, the browser extension for the wallet, some platforms can let you like literally create — sorry, mint an NFT — because you can create the artwork in any form you like, or whatever the object is, or digital asset or file.

      Now what happens after you mint it, what’s the next thing that happens? So you can put a name on it, you can specify program terms, you know what kind of royalties, different systems may have different options: Some of them themselves take 10%; others you can program in like as this increases in this much value; I’ve seen people do creative stuff like the artwork reveals itself the further you go along the bonding curve — like they’re doing creative stuff with the art itself kind of interacting, so it’s not just like a static piece of art that just happens to be going through this chain… What happens after the minting?

      Linda: You can do different things, depending on what you want to do with your NFT. So once you mint on these marketplaces, you can just have it listed, and try to share this link with other people, and have them bid on the piece of art. You can set a price; you can have people bid and then accept different bids.

      Or, you could just have this created for yourself: And, let’s say you want to make some worlds in a virtual land, and display your artwork in a virtual art gallery, and just place your artwork there…

      There’s all kinds of different things you can do with it if you think this piece is really valuable. I’ve actually seen people talk about swapping NFTs; so different artists are like swapping with each other. I’ve also seen people put up the art as collateral, and then get out a loan for it.

      So, you can really do whatever you want with this. But the most common, basic thing I’ve seen is just people selling their artwork, and someone purchasing it, and then maybe storing that like on their virtual land, or having it displayed on their profile. And there’s like a social element to it: People talk about it a lot, like hey I just purchased this piece of art, and they’ll talk about it on Twitter. And there’s an app that aggregates purchases from all of these different marketplaces, and displays it almost like an Instagram feed — so you can have like a social element to who’s purchasing what, who owns what, and have people form communities around it.

      Sonal: I’ve also often wondered if there will be like a Pinterest for NFTs, where even if you’re not the owner, you can kind of… “collect” it. Like one of the things that I use Pinterest for is it’s sometimes aspirational stuff I would never, ever buy — and just like having Pinterest boards is a way for me to “collect” it in a different way. Like I wonder if that would even happen, and if people would create, like, fees for doing that as well. I don’t know if you guys have seen that yet, but I wonder about that.

      Linda: I could see something like that happening, and, also just the fractionalization aspect to it — you can cut it up to really really really tiny pieces, and maybe people can own just really tiny elements of this piece as you’re looking through your Pinterest board or something.

      Ecosystem players

      Sonal: So, what you guys just described as all these different steps, there’s a whole ecosystem of players that have now emerged and are continuing to emerge. We’ve already named a few — like sites for showcasing an online collection; displays; like online galleries, we have curated galleries coming up; we have marketplaces; we have other tools for managing details… How would you break down the taxonomy — give me a map, and the terrain.

      Jesse: Yeah so there’s vertical marketplaces, right, where there’s marketplaces for like curated art or for certain types of collectibles. And then there’s horizontal marketplaces, like OpenSea, where that’s more of a search box for all NFTs. And the reason they can do that is all these NFTs live on the blockchain, it’s completely open. So they can query the blockchain and aggregate all of them. And you can find literally, pretty much every NFT on a horizontal marketplace like OpenSea.

      Sonal: Which is great, because not everybody knows how to interface with crypto. This is like the way the web itself evolved.

      Jesse: Right. And then, each of these kinds of marketplaces — both vertical and horizontal — more often than not, allow creators to mint, on the platform. Simple way to think about it: there’s both supply and demand: and you can either get it in the vertical form, which is specialized; or horizontal, which is sort of everything.

      One ofth really interesting phenomenon that’s happening on the demand-side of the market is you’re starting to see these really interesting collector… organizations sprout up: So, crypto makes it really easy to send value — like as easy as sending an email — as a result, people are pooling value in interesting ways in order to participate in this market. One really cool experiment is this thing called Flamingo DAO (DAO stands for decentralized autonomous organization) — the core idea is you can pool resources with anyone with a crypto wallet; send money into this smart contract, that acts as sort of like a bank account, and then that bank account can go and buy NFTs. The group can buy NFTs.

      And so what that kind of looks like is… maybe something like a fund, or you could call it a “gallery” that’s acquiring work. And by being in that collection, the creators’ gaining distribution to the audience of collectors/investors who pooled resources in the first place. So that’s another really interesting phenomenon that’s uniquely crypto enabled. And I think we’re gonna see a lot more of that.

      Sonal: I do too, and I looove that because one of my favorite things in the creator economy in general is the way collectives can emerge, both ephemerally and permanently (I have like a whole tweetstorm about this) — but I think it’s super powerful to think about what happens when you unlock that kind of coordination… Keep going.

      Jesse: Yah. I mentioned vertical and horizontal marketplaces; there’s also adjacent just media platforms — like we touched on Denis’s project Mirror — which is a blogging platform, where anyone can mint their blog posts as an NFT.

      And the question, why would you want to mint sort of a blog post or an essay as an NFT?

      Sonal: Yes, thank you for answering that!

      Jesse: <chuckles> If you’re an investigative journalist for example, there’s not a whole lot of great tools for you to monetize as an independent right now; subscription can be less conducive to long-form journalism. And what’s kinda cool about Mirror is — similarly to the prior idea of people pooling money — it allows for a writer’s audience, to pool resources in the form of a crowdfund: “Hey, I want to see this investigative report written, and here’s the money to do it.”

      And as a participant in that crowdfund, you don’t just become a patron of the creator, or the writer: You become an owner, a fractional owner, of the NFT that they produce when they publish that blog post. And, you can sort of analyze the psychology of one of these backers, but I think it boils down to two things: There’s the idea of *patronage*, right, you’re being a patron of this creator and helping them get the work done; but there’s also this vague notion that, in the future, if this piece becomes very valuable, I’ll be on the “cap table” of the post. Like Elon Musk published his famous blog post, the secret masterplan of Tesla. And just recently, you saw Jack, founder of Twitter, sell his first tweet (ever) as an NFT for millions of dollars.

      So, you can see this idea going a lot further: Where new, big ideas enter the world as blog posts, and people crowdfund those big ideas that they want to see happen in the world, and become part-owners in the blog posts — that becomes the sort of canonical representation of that idea.

      Sonal: I saw Dylan Field post a thread a couple of days ago that I thought was wonderful, talking about some of the extension of ideas around NFTs. He described like “proof of fandom” — and we have lingo in the crypto world of “proof of stake”, “proof of work”; and it was neat to have this idea of “proof of fandom” — it kind of ties back to this idea of monetizing moments as well.

      And in this case you’re talking about ways for creators to have their fans — and one thing we’ve talked a lot about on this podcast; I did a podcast with Kevin Kelly about how you can actually invert the model of payments, where it’s not a creator selling, but buyers and fans monetizing *their* attention — And so the idea of that is super interesting… because you can imagine fans and collectives like buying and owning these things.

      And Dylan even went so far to point out even “community as art” in that example, which I thought was super interesting.

      Linda: So, that’s an area that I’m really excited to see… I haven’t really seen too many people working on this yet, but, the idea of having so many DAO members being able to vote how this artwork looks and kind of have it be collective artwork.

      I was in this DAO called Saint Fame, and we would vote on different parameters of the design of clothing items, and then this DAO would manufacture them and ship them to people that purchased it. And so you had like this group of people deciding what the design looks like, and you can imagine that anyone can join these DAOs — it could even be anonymous people and from all over the world. And so you can collectively create or invest in things together, which is really exciting to me.

      Sonal: Connie Chan (our partner) has often talked about influencer monetization, and she talks a lot about what happens in China with livestreaming and how a lot of fans will ask their audience like “Should I wear this today, and do that?” And some of it can kind of veer on dystopian in some models, but in many ways, it’s also incredibly empowering that you can choose to monetize the things you want to and have models for doing it.

      But right now, it’s the platforms that take all the money. So what’s really interesting about what you just described is that you could essentially do the exact same thing — but in this sense, you’re creating not just artworks, but you’re actually creating collaborative decisions around… a person’s wardrobe, or a fashion line, or however they want to develop products (even physical products) based off of that. Which I think is super fun and interesting too.

      Jesse: Just one last thing to add there, along the lines of proof of fandom, is this idea I’ve been calling “Patronage Plus”: So in Web 2, it’s very easy to become a patron of an artist or creator whose work you admire, by subscribing to them on Substack or paying a subscription on Patreon. And what that essentially does is gives you access to their work, but it also allows being a supporter financially of the work itself.

      NFTs allow you to do the same thing: in some cases, the NFT can give you access to a Discord or a newsletter (and we touched on that) — but the *plus* part of patronage plus is what’s new and uniquely-enabled by digital ownership. And that is the possibility of being able to profit in the future from the resale of that ownership to someone else, maybe as the creator’s profile raises or the demand for their work grows.

      And, I think that “plus” is really key because it’s a very strong incentive to become a patron in the first place. So, patronage plus may end up growing the market way bigger than patronage that we saw in Web 2.

      Sonal: What’s so amazing about that is the golden age of art, many argue — like in the Renaissance era in Italy, Medici family, etc. — people argue that patronage in the first place is what unlocked that. And so what you’re describing is a more democratized form of patronage, and the “plus” is a way to really have this knock-on effect over time — which is really investing and democratizing — in a way that is accessible, to everybody. Because it’s not just the rich Renaissance families that can do the funding of the arts.

      Jesse: Yeah, it mixes patronage with capitalism.

      Linda: Yeah. So there’s a DAO called Yield Guild Games that I’ve been participating and active in, and there’ve been people in the Philippines who have been earning a living wage playing — like Jesse talked about you have these DAOs being able to own NFTs — and what they do, they’re a DAO of gamers: gamers from all over the world who are participating in these blockchain games.

      And there’s one really popular game called Axie Infinity — you have these like Pokemon-like creatures that battle each other, and each of these are NFTs — and you can battle and earn currency in this game. And sometimes these Pokemon creatures, like they might be too expensive because they’re so valuable. And so what we’ve actually seen in this DAO is players within this DAO leasing out NFTs to other players. It’s a really cool collective of people being able to join this group of gamers. And one thing that they’re doing right now is this DAO is investing in virtual land in the games that they’re playing, because they’re experts in these games themselves. And they’re actually developing like the land in these games as if…just in like a physical world of developing real estate and making it better (the idea is that they’re going to be just owning tons and tons of virtual land).

      Sonal: One quick thing — again, a DAO is a decentralized autonomous organization; people have often talked about cryptoeconomies over time enabling these sort of organizations because the history of the firm is very much entrenched in a physical world, not a digital world — but why do these things have to exist as a DAO; what’s the point of that? I’m asking because I’d want to know like, why a DAO specifically.

      Linda: So I don’t think everything has to be a DAO; there are plenty of times where a company makes a lot more sense.

      But, what’s really interesting about DAOs is there is a lot of more transparency — and so the funds that are managed by the DAO, it’s completely transparent where funds are being moved to and from, anyone can view the balance at any point in time. As you can imagine, a traditional company, you don’t have access to the balance sheet at all periods of times; and oftentimes, they’re just maybe released on a quarterly or annual basis.

      So the DAOs even the playing field, create more transparency; there’s lower barriers to entry in a lot of cases: You don’t even have to reveal your identity; it’d be really hard to join a company where no one knows who you actually are. That just fits very closely with the ethos of crypto of: global, open, kind of nature. <Jesse: Yup>

      Sonal: And by the way to be very clear, we don’t mean identity as in like anonymous, because you’re pseudonymous technically; like people can actually trace WHO you are without actually knowing who you are.

      The energy question

      So, now I’m going to have you guys break down even more misconceptions for me — like we talked about “just a JPEG, what’s so unique about a JPEG” — but there’s actually a lot more misconceptions, especially given recent buzz, all this commentary about “the energy, the energy, like minting is all this energy”.

      This is obviously an artifact of people thinking in terms of Bitcoin, which can be energy intensive; so, can you help clarify the energy question when it comes to NFTs?

      Linda: Well I think that there are a lot of misconceptions around that. So yeah, proof of work does require energy, but not every blockchain is proof of work. Proof of work involves physical miners actually verifying that these transactions happened. And so it’s just really energy intensive because you have to prove that you’re expending some sort of work, to produce this output.

      And so in Ethereum’s case, they’re migrating from proof of work to proof of stake — which is kind of equivalent of virtual mining – so, rather than spending let’s say $1000 on mining equipment, you’re taking that $1000 and locking it up into the system, and being randomly selected to verify based off the capital that you put in. So it’s just a virtual aspect of mining, and you don’t have to have the physical ones expanding energy.

      And then increasingly, we’re also having more movement towards Layer 2, like protocols built on top of Ethereum. Because people do want the be less energy intensive when it comes to verifying ownership on a blockchain. So there’s going to be less of that narrative, I think, going forward.

      Jesse: Even proof of work mining gets more of a bad rap than it deserves; it’s certainly true that it consumes a lot of energy. However, a lot of the miners who are doing the proof of work locate in areas where there is latent capacity — so renewable sources of energy that are untapped, for example like hydropower; there’s excess demand, well then it goes into mining. Like for example, there’s like natural gas emissions from oil fields; and that’s gas that is otherwise, it’s going into the atmosphere, but instead can be burned to produce proof of work proofs and earn Bitcoin.

      I’m not, you know, defending this practice. But I’m just noting that a lot of these emissions are either sort of latent, OR, a large part of the energy mix of proof of work mining is from renewables. And that again, is part of the conversation that’s under-discussed.

      Sonal: I am so glad you brought it up because the whole point of the show, again, is to give the nuance, that may or may not exist out there.

      Jesse: I also think a lot of the noise out there about the energy consumption of NFTs, really fails to take in the sort of relative measure of energy consumption more broadly.

      So, if you think about something like Art Basel, there’s a lot of very rich collectors who fly private to Art Basel every year in order to collect work. And I don’t know what the emissions of all those private jets is, but I would expect it’s a lot of CO2. And so to get into the game of quantifying the specific emissions of an artist’s work, I think is a very complex topic that’s sort of under-appreciated in 140 or 280 characters on Twitter when you say “Oh, this NFT caused X amount of CO2 emissions.” Well, what about all the freeports, what about all the private jets flying to Basel every year?

      So it’s a very nuanced topic, and I don’t think it’s fair to creators who are just using these new tools — which are becoming more and more efficient — to shut it down on the basis of this very headline-grabbing, relative value measure.

      Sonal: That’s fantastic. By the way I have been to Art Basel Miami, not the one in Switzerland, in 2006; I did not fly in a private jet, I was a grad student, I did not have that much money — but yes, I agree with you, that it would be very slippery slope.

      Linda: I also saw a tweet by Andrew Steinwold saying that actually, these digital art[s] are actually really environmentally friendly in that you’re not buying all these like physical supplies of like cotton for canvases, and wood for easels, and oil… And then you have all these shipping costs of moving this artwork to other people.

      There is always going to be aspects to anything that’s created, that you can always analyze and look at what is not environmentally friendly about it.

      Sonal: You’re absolutely right, and in fact, one of my absolute favorite artists, I bought a painting from her. I went to her show in New Orleans; I flew. She shipped the art to me afterward, it was such a process trying to bring it here, and the shipping — just even the materials to pack it, like all of it, it was intense and very complicated — and in fact, I had to hire someone to help me open the crate because it’s like screwed in, in wood. It was like not even possib- there was energy used to like take the thing out of this box. <Jesse chuckles> So, I agree with you. It’s a very tricky game to start comparing on that front.

      “Permissionless” innovation

      Okay! This idea of “permissionless”, you used that phrase a lot. If I were like a regulator and hearing that, I would freak out and be like, “Permissionless?! That means all kinds of bad behavior and actors and blah blah blah.” How would you address the concerns about things being permissionless — or even this idea that you know, you can’t even recover your key if it’s lost — there’s not like someone who’s holding that for you.

      Jesse: We can define it in the same way like cash is permissionless, right. Again, the wallet analogy is useful because if you lose your wallet, chances are you lose your cash and it’s gone. And similarly in the cash economy, you can buy all kinds of goods; they can be illegal goods, or they can be perfectly normal goods, and cash is used for both things. And so the same is true of cryptocurrencies, and I think the same is true of NFTs.

      There’s going to be bad actors right, there’s going to be people infringing on other people’s IP — and you know the legal system is going to have to step in and address those issues. However the benefits I think far outweigh any sort of negative or nefarious uses of the technology in that any developer can build new experiences around the way we consume media… again, when you contrast today where only the developers who work at Facebook or only the developers who work at Twitter can experiment, and innovate on the information that we see on those platforms… I think we’re in a much healthier state if EVERY developer in the world is free to innovate in a open way, without having to ask permission of these big platforms.

      Sonal: Right, that’s what you mean by permissionless. And by the way, it’s worth noting all those examples you cited — copyright infringement, IP, etc. — that’s pretty prevalent in the physical world, and you don’t often always have recourse (unless you’re Getty in doing this ridiculous royalty and provenance tracking).

      And, we’re talking about you actually have the solution baked into the very problem in the system here.

      Jesse: Yeah, in one sense, you could argue that blockchains actually make the job of forensics a lot easier because all the information is publicly accessible and available to anyone.

      Sonal: Our partner Katie Haun would obviously argue for that argument; I mean, at the DOJ that’s literally what she did!

      Jesse: Right, it’s all about finding the right balance where the bad actors can be addressed, and meanwhile the good actors are free to innovate.

      Corporate innovation & NFTs

      Sonal: Okay, so last thing. Can you guys give some just super quick practical considerations for startups or industry? I’d love to particularly hit mindsets, it doesn’t have to even be advice — for people who are consumers, people who are creators, and even institutional players.

      Linda: I find that just having conversations, and kind of plugging yourself into communities, and building in public is always really great to do in the crypto space. This space is still really early on, and people are all trying to figure everything out. So no one is a complete expert on what’s happening in NFTs and everyone’s very open to talking, collaborating. So never be afraid of asking questions; joining different communities on Discord, or Twitter, or wherever they’re chatting.

      Sonal: Big corporates and the big institutional banks and big DeFi players like banking and traditional players — they’re not the types to go into a Discord and try things out or have the mindsets you outlined. Any thoughts there, for that group?

      Linda: Traditional institutions can consider how NFTs could make things more efficient for themselves — so having these financial assets that everyone has to keep track of, might be really inefficient or costly. NFTs will enable this to just be a lot smoother of a process for them.

      So it may be worth looking into research — it doesn’t have to be digital art that they’re turning into NFTs; it could also just be unique financial assets that they have to manage themselves.

      Jesse: Yeah, big corporates and others participate in the markets for creative work, through various channels, a lot of companies work with creators and influencers on marketing and distribution. And NFTs offer a new channel for both of those things, right?

      I also think that, coming to be an owner of a creator/influencer’s work will be another way to gain their attention and potentially gain distribution through the lens of marketing. And that’s kind of an interesting idea.

      Sonal: One of the ones that I find very compelling is a new definition of employees in a modern era, where employees can be creators while working for a company and kind of get more ownership of their ideas — ‘cause traditionally it’s like very binary model; there’s not like a middle ground — and I wonder if that’s going to evolve through NFTs within companies, and even extending outside the borders of companies like in a classic open-innovation model.

      Jesse: So I think you’re touching on a really big idea — I would describe that as the ownership economy — where, in Silicon Valley, employees at startups get equity in a startup to align their incentives with the success of the company. And that model has worked really well for attracting the best talent to kind of work at startups. But it’s not been accessible to everyone, right, and as a result, the talent pool is not as big as it could be. And crypto kind of changes the game in that now it’s possible to send ownership value — whether an NFT or ownership of the bitcoin network — which you can now send that anywhere in the world instantly. And as a result, you can make ANYone an owner on the internet.

      And so I think this is a really profound idea where, it’s going to change the way that people come to work, in that they won’t have to go and become a full-time employee to earn some ownership value for the value that they contribute to the platform or service that they’re building or consuming.

      Sonal: Which reminds me of course of that famous Bill Joy quote that we all love which is that the smartest people in the world won’t ever all work for you. So, if you’re gonna embrace open innovation, open source, or extend your talent pool, that is a great way to give those employees quote-“ownership” — even if it’s fractional ownership, which is great.

      Jesse: Yeah, and NFTs make it accessible to everyone, not just technologists but consumers and creators as well.

      Sonal: Awesome you guys. So on this show — even though this a 3x explainer episode — I ask people to kind of give me a quick, short, you know “what’s your bottom-line” on this whole theme; give it to me.

      Jesse: I have a media background, so I love to fixate on a future where literally, every piece of media is incepted as an NFT — I’ve used this term a few times in the discussion, but I think what we’re building here is this universal library of media that’s programmable and where value flow is baked into the technology itself. And that’s just going to lead to a renaissance in online creativity, where the creators of the work are remunerated more fairly than they have in the sort of Web2 era.

      Linda: Yeah, there’s a lot of really exciting stuff happening in the NFT art space, and we have so many creative people coming in and it’s going to make crypto overall much better. But NFTs are also applicable to many industries where you track ownership, and currently have a middleman taking fees for that service. So, I expect there to be NFTs in all kinds of different industries like gaming and finance and healthcare and all kinds of other areas.

      Sonal: It’s an inevitable story of technology that you give people tools and things will happen — so it’ll be interesting to see what happens when we unlock that human ingenuity. You guys, thank you so much for joining this week’s episode, this 3x explainer episode of “16 Minutes”. Thank you so much, Linda and Jesse.

      Jesse: Thanks for having us.

      Linda: Thank you.

      • Jesse Walden is the founder and managing partner of Variant Fund, investing in crypto networks and founders. Previously, he was a partner at a16z Crypto and cofounder of Mediachain (acquired by Spotify).

      • Linda Xie is the co-founder of Scalar Capital, a crypto investment firm. She was previously a product manager at Coinbase and a portfolio risk analyst at AIG.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Semiconductor Shortage and the Global Supply Chain Squeeze

      Frank Chen and Zoran Basich

      In this week’s episode of 16 Minutes, our show where we talk about tech trends in the news, what’s hype/ what’s real, and where we are on the long arc of innovation, the topic is semiconductors – specifically, the ongoing global shortage that began last summer and has intensified in recent weeks. So much so, that the U.S. president signed an executive order just last week to address concerns around the shortage, calling for reviews of supply chains for critical sectors of the economy.

      Our expert is a16z Operating Partner Frank Chen, who led our research arm and has also joined past episodes about semiconductors on this show including one with Steven Sinofsky and Sonal in which they analyzed the ARM and Nvidia news.

      Frank joins a16z’s Zoran Basich to cover the bigger picture of the chip shortage including geopolitics, the pandemic, and complex worldwide supply-chain dynamics — all in almost exactly 16 minutes!

      Show Notes

      • How the semiconductor supply chain works [1:14], what caused the current shortage [3:10], and errors in forecasting that some companies made [4:49]
      • The impact of demand (types of chips needed by various industries) [5:54] and supply (cost of building factories, COVID disruptions, and the scarcity of raw materials) [8:24]
      • Geopolitical questions involved [11:55], including competition with China [13:58]
      • Thoughts about how the shortage will play out [15:14]


      Zoran: Welcome to this week’s episode of “16 Minutes,” our show where we talk about tech trends in the news, what’s hype, what’s real, and where we are in the long arc of innovation. The topic today is semiconductors, a topic that has been in the news a lot in different forms. Specifically, we discuss the ongoing global shortage that began last summer and has intensified in recent weeks. So much so that the U.S. president signed an executive order to address concerns around the shortage, calling for reviews of supply chains for critical sectors of the economy. 

      Joining us as our expert is a16z Operating Partner Frank Chen, who led our research arm, and has also joined past episodes about semiconductors on the show, including one with Steven Sinofsky and Sonal, in which they analyzed the Arm and Nvidia news.

      In this episode, we’ll cover the bigger picture of the chip shortage, including geopolitics, the pandemic, and several other factors, all in almost exactly 16 minutes. But first, Frank covers the immediate impact on the automotive industry, which was highlighted in the news of the executive order.

      The semiconductor supply chain

      Frank: A bunch of auto manufacturers, this includes GM, and Ford, and what used to be called Fiat Chrysler, now called Stellantis, all of these companies have basically shut down or slowed down their production of cars because they can’t get enough chips. There are very calibrated supply chains, tens of thousands of suppliers. And right now, what’s happening is, you know, Ford’s F-150, the best selling truck, is held hostage to a $0.25 cent semiconductor that all of a sudden they can’t get enough of.

      Zoran: So this news is really closely tied to the supply chain. That’s where the White House is focusing. Help us understand the landscape here, Frank. Who are all the players involved up and down the chain and how are they interrelated?

      Frank: The way that chips flow is that you have semiconductor vendors like Nvidia, and NXP, and Renesis, and Panasonic, Toshiba, Samsung, TI, and so on, they make the chips. Some of these companies make their own chips like Samsung and Toshiba, so they’re called integrated device manufacturers. Other companies like Qualcomm and Nvidia design the chips, but then use other companies like TSMC, or UMC, or Samsung, to actually make the chips. These design-only companies are called fabless chip companies. But in either case, you take the chips, you sell them to companies called tier one electronic systems providers. So these companies have names like Bosch, and Delphi, and Harman, and Denso, and Siemens, Continental. And then these companies in turn sell to what’s called the OEMs, the car manufacturers. That’s what we’d recognize as a car manufacturer. So a GM, a Ford, a Tesla, an Audi, a BMW, so on and so forth. Right? So that’s sort of the supply chain.

      You got a chip company selling to an electronic systems company. They might make the onboard entertainment system, they might make the anti-lock brake computer, or they might make the adaptive cruise control system. And then the OEMs assemble them into cars. And so, there is a lot of complexity.

      Causes of the current shortage

      Zoran: So how did the automakers get into this mess? What are some of the factors that caused this chip shortage?

      Frank: Let’s start with the demand side. So, three things. First, software is eating cars, which is to say that the percentage of a car that is electronics has been steadily increasing. So it used to be a car was rubber, plus glass, plus steel, and that was pretty much it. But now, in 2020, electronics are about 40% of the cost of a car. For a point of reference, in 2000, it was probably 18%. And, you know, these days, as you think about all of the cool safety features, right? Somebody’s in your blind zone, or adaptive cruise control, or automated stopping if there’s a pedestrian in front of you, right? All of those are obviously electronics.

      The second thing is, this was a classic case of under-forecasting. So what happened was, COVID hit, and everybody battened down the hatches. Everybody was like, oh, car demand is going to drop off the cliff here. And so we better be conservative and cut back all our orders. And what happened roughly at the same time was, there was a set of things that people wanted that demand went the other way, right? So think of Chromebooks, and laptops, and webcams, and everything that makes Zoom possible. All of this work at home stuff created a lot of demand for TVs, and computers, and so on. And so, the demand for semiconductors to feed those things sort of leapt into that vacuum that the car guys left. And then once we figured out that car sales weren’t going to be as dramatically impacted by COVID, they went back to the suppliers, and then they discovered the supply is not available. In other words, they got spoken for by the computers, and the cell phones, and the webcams, and IoT devices.

      Zoran: So there have been lots of headlines about this chip shortage, lots of angles to unpack, but let’s home in on the pandemic for a minute. How and why did this under-forecasting happen? And why specifically is that a problem when it comes to chips?

      Frank: I think it was human nature. Which is, COVID happened and you were like, gee, when will demand really ever come back to normal? And I don’t want to be the guy that over-ordered everything, right? Like, once the TPS reports come out, I’m a complete outlier because what kind of idiot would I be, if COVID happened and I was the guy who over-ordered everything by two orders of magnitude. The other thing is, it’s very long lead time to spin up a new semiconductor, in other words, to change the line so that it’s making your chip versus somebody else’s chip. This is measured in tens to dozens of weeks, right? So, 30 weeks. And so, you can’t turn it off and turn it on, on a dime, right? And so, when the orders got cancelled, the lines got retooled to build other chips. And so, to turn the line back on to build your chip, we could be talking the better part of a year, which is why most people think that the shortage is going to be another couple quarters before we’re done with it.

      Impact of increased demand

      Zoran: Okay, so on the demand side, you’ve mentioned the rise of electronics and cars. We talked about the forecasting mistakes. Let’s get a bit more into what’s hype and what’s real here. What else do we need to know to make sense of this?

      Frank: On the scale of things, car manufacturers aren’t the biggest customers for chips. So, by far, the biggest customer for chips is cell phone makers. Like, when Apple places an order, that’s an order, right? Like, that’s tens of millions, hundreds of millions, billions of components. And so, like, they’re more reliable as a customer. You can sort of see why if you are a chip manufacturer, like a Panasonic, or Toshiba, or Infineon, or STMicro, or TI — when Apple, or Samsung, or Dell, or HP comes to you and says, “I need more chips.” You say, “Of course. When do you need them?” Right? And then Ford, and Jaguar, and, you know, all of the other guys have to sort of kind of wait in line.

      And then the other thing that is true about car chips is, they can be harder to make, which is, their temperature range, their operating lifetime, the failure rates of these chips, right? So it’s like one thing for your Chromebook’s light sensor to go bad, it’s another thing altogether if, like, the radar that powers your adaptive cruise control goes bad. Like, that thing can’t go bad, right? Because you’re going to crash into somebody.

      Zoran: Yeah. And some of the chips that are used in cars are older generation chips. So, because they’re more expensive on the consumer electronics side, that’s another reason that they’re seen as more higher priority customers.

      Frank: So, yeah, two classes of chips, the older stuff, right, for the anti-lock brakes, and then the newer stuff for what the car industry roughly calls ADAS, right? So, the advanced driver assistance systems. One way to measure this is, what’s the nanometer process technology used to create it? And we’re headed towards seven, six, five nanometers. Think of that as sort of how fine is the etching on a semiconductor that defines the circuit path, right? And car manufacturers — some of their chips, you don’t need [a] 7, or 6, or 5 nanometer, you’re fine at 180. But the second class of chips that the car companies are increasingly buying are basically the exact same chips that go into a smartphone core, right? So if you think about machine learning, as we head towards autonomous, you know, they will be the most advanced chips, with a ton of transistors on them to do linear algebra, because that’s what machine learning demands. And so, this sort of emerging class of chips are exactly the same set of chips.

      Issues with global supply

      Zoran: Okay, so that’s the demand side. When the pandemic hit, automakers pulled back on their orders, while at the same time, demand was rising for consumer electronics. And the chip manufacturers turned to that segment of their customer base, and started producing for them. And it’s very hard to stop on a dime, and then take the automakers’ calls, who now suddenly are calling you and saying, “Hey, we want to restart our production.” That all takes a long time. So, let’s talk about supply. What happened on that side of the equation?

      Frank: So it turns out that the world has a fixed amount of manufacturing capability for semiconductors, which kind of seems bizarre, because we all know that software is eating the world. And, you know, the world has nearly infinite demand for chips. Most semiconductors are made outside of the United States, despite the fact that the United States dominates revenues for semiconductor design, through companies like Intel. But most of the fabs, which is the factories that make semiconductors, are overseas. But it turns out these fabs are incredibly expensive and relatively low margin to build. Think of a factory that might cost $10 billion to build, and it’s obsolete in five years. The rate at which semiconductor fabrication changes is so fast that, like, all of the equipment that you just bought from Applied Materials and JLA-Tencor, like, that thing is going to be obsolete in five years.

      And so you have these incredibly expensive factories that depreciate very, very fast. The equipment in it sort of needs to get replenished very, very quickly. And so, the industry’s kind of rewarded companies that are called fabless designers. In other words, they’re companies that design chips, but don’t actually make them. So the fabless designers that we all know are companies like Qualcomm, and Broadcom, AMD, Nvidia, Apple itself, right — all of the, like, super awesome chips that they design, they don’t actually make. They go to semiconductor fabs, the largest of which is the Taiwan Semiconductor Company, that actually operates the factories. The top three countries that actually make chips, South Korea, Taiwan, and Japan, they have all of the factories because fabless is rewarded by the investing community. You’d much rather have the higher profit margins of a Qualcomm than the lower profit margins of a TSMC, the Taiwan Semiconductor Company.

      Zoran: Okay, so you have this issue of how incredibly expensive it is and how frequently you have to accommodate the new technology they have to build. What are some of the other supply factors?

      Frank: Well, like every other manufacturing facility, the semiconductor industry got hit by COVID itself. People need to be in the factories feeding the wafers, and doing quality control. And so there was some slowdown as a result of that. Now, the good news is, South Korea and Taiwan are the biggest manufacturers, and they were much less exposed to COVID because they were very aggressive with their lockdowns. But there are other things about semiconductor manufacturing. So, Taiwan is going through a drought, just like California went through a drought. And it turns out, you need a lot of water to make chips. TSMC’s daily water consumption is 156,000 tons a day. In the northern part of Taiwan, where these factories sit, that amount of water is 10% of the region’s daily supply of water.

      And so, you have all of these weird supply things. You have, like, raw material shortages. We’ve got the COVID hit. We’ve got the very, very small number of fabs, just because they’re expensive, and sort of the financial community willingness to fund these super expensive factories is sort of low.

      Geopolitics of semiconductors

      Zoran: So we’ve talked about the factors like drought, the economics of chip production, the pandemic, obviously. Now, where do the geopolitics come in? How big a deal is that, really?

      Frank: Yeah. So, geopolitics is a big deal. So, you’ll remember that the Trump administration basically forbid American companies from buying from Huawei. And then they later extended it to, you know, requiring a license to sell to Huawei. And then there are licenses that the semiconductor tool chain now has to apply for in order to sell to Chinese companies. And so, if there were no politics, you would do what manufacturing has done for the last four decades, which is, you’d fire up China. You’d make it possible for Chinese fabs, not Taiwanese ones, not South Korean ones, not Japanese ones, to bloom. But the problem there is that the United States, rightfully so, wants to be a little careful about what kind of semiconductor manufacturing equipment they will sell. Because the worry is that, they’ll buy it, they’ll reverse engineer it, they’ll infringe on the intellectual property. And lo and behold, they can make their own semiconductors. And by the way, that is China’s explicit goal, which is that they want to have the number one semiconductor design and manufacturing industry in the world by 2030.

      And so, now the whole thing is a geopolitical dance. Like, you could imagine any administration, this is not a red or a blue issue, saying, like, we’re not sure that we want to sell China the equipment to make the most advanced semiconductors.

      Zoran: It’s interesting because, you know, there’s this perfect storm that happened with the pandemic, and also some other factors, which we haven’t mentioned. You know, there were a couple of fires in Japanese factories that had a negative impact on supply. Even in Texas, there were a couple of factories that because of the recent cold spell, had to shut down for some time. So all these things kind of just built upon each other to create this somewhat perfect storm. And so that created this instant problem, but it also highlighted this larger problem [that] needs to be addressed. So what’s the bigger picture about what this means for innovation, given these geopolitical pressures?

      Frank: Yeah. I mean, one of the big things that the industry is asking itself is, you know, China’s ambition to be number one, can they get there? And they don’t want to stop at just the chips, right? They want to be the complete, fully vertically integrated stack. So if you think of the vertically integrated stack, it’s sort of iOS running on top of a bunch of chips, with iOS. Or if you think about Windows, running on Intel. Like, in the next 10 years, will there be a Chinese operating system running on a Chinese chip, right, with all of the motherboard, etc., etc., design done by China. And if that were the case, will we really have two competing world ecosystems? It kind of reminds me of, like, the early days of communication. There was Docomo in Japan, and then there was Minitel in France. And like, they didn’t talk to each other. They were just little islands. And we knew that that world wasn’t good. What the world wanted was the internet, the connection of networks.

      But if we go back to this sort of geopolitically motivated desire to have your country own the factors of production in a completely integrated vertical stack, hardware and software, then we might go back to the bad old days, where compatibility was hard. And, you know, we sort of were kind of wasting “R&D” dollars building the exact same thing, just in slightly different ways.

      When the shortage may end

      Zoran: So we have the White House, President Biden, calling for this review. Short term, what’s going to happen here? And how long will this last?

      Frank: So most people are forecasting a couple more months, maybe quarters of pain. I don’t think, unless something surprising happens, I don’t think this is going to last that long.

      Zoran: I thought the solution was more manufacturing facilities need to be built in order for this truly to become solved. So how is this going to solve in the short term?

      Frank: You know, I don’t think it’s going to take, like, a brand new fab to unlock the current snarl that we’re in. So in the short term, you know, look, we’re not going to be at post-COVID highs on webcam orders forever, right? Like, they will go back to normal. And so the heat on alternative demand will sort of cool some. And then, you know, we’ve got the automaker chips in the queue now, right? And so, like, eventually, things will sort out. We’ve always had component shocks in the tech ecosystem. It’s just every now and then, we’ll hit a bad one. This one’s a pretty bad one because we have so many car factories making very, very expensive products, stymied by their $1 semiconductor being missing.

      Zoran: And that brings us right back to the news, full circle. So, let’s go to our bottom line, Frank. What are your takeaways and final thoughts on the topic?

      Frank: We have some soul searching to do about what the shape of our supply chains ought to look like and how much they should reflect the geopolitics of the time, or if technology wants to and should be a country-independent thing. Where, you know, the best ideas, meritocracy, take the day, as opposed to, we’re going to have the U.S.-led tech stack, and then a Chinese-led tech stack, and then they don’t really speak with each other, and view each other with mutual suspicion. The supply chain for technology has always had shocks. Because software is eating everything, these shocks are now rippling beyond technology.

      Zoran: Frank, thanks so much for being with us today.

      Frank: All right. Thanks, Zoran.

      • Frank Chen is an operating partner at a16z where he oversees the Talent x Opportunity Initiative. Prior to TxO, Frank ran the deal and research team at the firm.

      • Zoran Basich is an editor at a16z & Future, focusing on crypto and corporate development/ finance. Previously he covered venture capital and the startup ecosystem at the Wall Street Journal and Dow Jones, and was the banking editor at NerdWallet.

      Assembling an Egg

      Vineeta Agarwala, Justin Larkin, Judy Savitskaya, and Lauren Richardson

      On this episode of the Bio Eats World Journal Club, we explore the very compelling question of whether we can use our understanding of developmental biology to create oocytes (aka eggs or female gametes) from stem cells in the lab. If possible, this could be on par with the development of in vitro fertilization in terms of extending fertility. But creating an oocyte from a stem cell has some unique and high-stakes challenges. Host Lauren Richardson is joined by a16z general partner Vineeta Agarwala and deal partners Judy Savitskaya and Justin Larkin to discuss the research article “Reconstitution of the oocyte transcriptional network with transcription factors” by Nobuhiko Hamazaki, Hirohisa Kyogoku, Hiromitsu Araki, Fumihito Miura, Chisako Horikawa, Norio Hamada, So Shimamoto, Orie Hikabe, Kinichi Nakashima, Tomoya S. Kitajima, Takashi Ito, Harry G. Leitch and Katsuhiko Hayashi, published in Nature, which makes a big step towards this goal. The conversation covers which aspects of oocyte biology the authors were able to replicate, which they were not, and where we think this field might be heading.

      Show Notes

      • A brief discussion of the biology [3:24] and terminology [5:27]
      • Potential uses for oocyte biology [7:34], what it would take to convert cell types [10:36], and the development of the oocyte [12:36]
      • Limitations of this study [16:59] and the future of oocyte biology [19:25]


      Hanne: Hi, I’m Hanne.

      Lauren: And I’m Lauren, and this is the Bio Eats World Journal Club, where we discuss breakthrough scientific research, the new opportunities it presents, and how to take it from paper to practice.

      Hanne: So, Lauren, you’ve titled this episode “Assembling an Egg,” but I’m going to guess we aren’t discussing your favorite breakfast recipes today.

      Lauren: Hanne, you know me well, nope. Today, we are exploring the very compelling question of whether we can use our understanding of developmental biology to create oocytes, AKA eggs — you know, the female counterpart of sperm — from stem cells in the lab.

      Hanne: Okay. So, refresh my memory. Do we have stem cells in our adult human bodies that could be used to turn into eggs?

      Lauren: Yes and no. Adults have stem cells, but their ability to turn into other types of cells is limited. For example, we have stem cells in our bone marrow, but they can only produce the different types of blood cells, but scientists now know how to turn some of our cells back into stem cells, which are called induced pluripotent stem cells, or iPS cells. If possible, this adult cell to iPS cell to egg cell transformation could be on par with the development of in vitro fertilization in terms of extending fertility, but creating an oocyte by this path is tricky. So, I’ve gathered some of our colleagues, a16z general partner Vineeta Agarwala, and deal partners Judy Savitskaya and Justin Larkin, to discuss a recent research article published in Nature, by Hamazaki et al., that makes a big step towards this goal. We discuss what aspects of oocyte biology the authors were able to replicate, which they were not, and where we think this field might be heading. We start with Vineeta describing the state of innovation in fertility. 

      Vineeta: Fertility, even just within the U.S., is approaching a $10 billion industry, and the majority of innovation that we see happening in the space is related to care delivery. How can we expand access to fertility treatments to couples who need them? That’s an important problem, but at the same time, it makes us wonder whether there are biological breakthroughs that could change the face of the industry even more than care delivery technology might.

      Justin: Yeah. For so many aspects of reproductive therapies and infertility challenges, the egg or the oocyte is often the critical limiting step, both from an ability to actually get to a healthy pregnancy, but also in the process of the patient journey. Having seen numerous people go through the IVF process and other kinds of iterations of IVF, it’s often just so hard from a patient journey perspective — literally being painful, takes lots of time, it’s expensive. And so, the thought of being able to fundamentally change access to oocytes, and eggs that potentially could be grown in vitro in ways that cut the process down and make it more affordable, has the potential, I think, to not only open up reproductive technology and therapies to a broader population, but to make that care experience just fundamentally different for patients. And so, seeing the early innings of that potentially be suggested in this paper was fascinating to me.

      Judy: Vineeta and I were actually pretty surprised the other day when we realized that the most expensive part of IVF, the single biggest line item, is the drugs that are required for egg retrieval. So, this is a way to just get around that entire process.

      Overview of oocyte biology

      Lauren:  Right. Could you unpack that? How is this solving the problem?

      Vineeta: Women are born with a couple of million potential eggs, and as we age, we lose those eggs, and we don’t make more. And this paper provides this suggestion of a way to engineer, from pluripotent stem cells, oocytes that can potentially be fertilized and give rise to a new being, without being dependent on that very constrained, constantly dwindling supply of oocytes that we’re born with as women.

      Lauren: Right. I think that is such an interesting idea, that when your baby is still in you, it has its full complement of eggs that it’s ever going to have. Females don’t make additional eggs after birth, so you have to work with what you have. As Judy said, it’s very expensive to retrieve eggs from a woman. So, this paper asks a really tantalizing question of — can we make new eggs from cells after birth, from your own cells after you’ve been born?

      Vineeta: I would think of it as a replacement for egg freezing, is one way to contextualize where this could, hypothetically, fit into the industry. Today, women are freezing eggs in their 20s so that they may access them in their 30s and 40s or beyond, and this would provide an alternative to having to undertake that process of retrieval, freezing, and hoping that the eggs are usable.

      Justin: Often in those retrievals, you’re only able to get — you know, let’s call it 1, 2, 3, sometimes if you’re lucky, more than that — but a limited number of oocytes, and so you have a limited number of shots on goal on potentially having a successful pregnancy. Where I think what got me excited about this paper is this idea of potentially having more opportunities [for] a successful pregnancy that aren’t as constrained by the cost and kind of care experience today that drives the typical IVF process that we alluded to earlier.

      Lauren: We’ve thrown a couple different terms around. We’ve thrown around oocyte, we’ve talked about eggs. Let’s define this. What is an oocyte, exactly?

      Justin: Yeah. An oocyte is not a static thing, and it goes through a number of different developmental phases, starting from an undifferentiated cell, ultimately to being a true oocyte, which is what we would call a gamete or female sex cell. 

      The journey to get there is quite a fascinating one. As you alluded to, it starts, actually, in utero. Then as it gets further along in its development, it starts out as what we call a primary oocyte. And as a primary oocyte, it still has the deployed genetic material. Then it starts meiosis, which is the process of going from being deployed and having two copies of chromosomes, to being haploid or just having a single copy. And in that process, it actually arrests and will stay in that arrested state until that female hits puberty, and then that full mitotic process isn’t completed until after ovulation and even with fertilization. 

      So, it’s highly dynamic, and I think the critical piece here is that there’s a lot of gene regulation that’s going on throughout, which opens up the opportunity for research studies like this to really define what are the regulators of that process. Can we recapitulate that in vitro setting to potentially have some of the applications that we talked about earlier?

      Lauren: Yeah. I definitely was doing my background reading in preparation for this conversation, and really appreciating all those different steps that happen during — in utero. What happens before puberty, what happens during puberty, what happens during ovulation, what happens at fertilization — there’s all of these different steps, all these different developmental changes that all of them seem to carry the name oocyte.

      Justin: One other nuance that I think is important, too, is just that the oocyte in isolation isn’t necessarily enough, right? There has to be what they call a follicle, which is a set of cells that surrounds it. It’s very hormonally active, a lot of cell signaling and transcription factors are produced in that process. And we start to see early recapitulations of that in this study. But in vivo, inside of the female, there’s a lot of other, kind of, cellular activity that goes on around the oocyte cell itself that has a lot of hormonal impact downstream.

      Potential uses in healthcare

      Lauren: Yeah. That’s a really good point, that it’s very dependent and has a very complex interaction with the ovary and the cells of the ovary. So, on previous episodes of “Journal Club,” we have talked about converting one cell type to another. For example, we’ve talked about converting stem cells into the cells of the pancreas that produce insulin. But what are some of the particular hurdles in creating an oocyte that you wouldn’t have to deal with in just converting a stem cell into, say, a pancreatic cell?

      Vineeta: One way to think about the second-order challenge here is that you have to not only get to a primary oocyte, which has grown and has the right differentiation to at least have started on the path of egg generation, so to speak. But beyond being a primary oocyte, you have to get to a secondary oocyte. And the difference is that you have to undergo the whole process of meiosis, which is how we generate genetic diversity. 

      And so, that secondary oocyte is a pretty complicated thing to make. Not only because its genome has to have halved in a very unique way, by a very unique process, but a lot of epigenetic signaling that then goes on to determine gene expression networks in the subsequent fertilized embryo are thought to be driven by a program set in that oocyte. So, some genes are expressed off of DNA you got from your mom, some genes are expressed off of DNA you got from your dad, and not vice versa. A lot of that epigenetic programming is thought to stem from a pattern that’s encoded in the secondary oocyte.

      Judy: Another major difference between this and other cell types that you could potentially reprogram is just the morphology. So, oocytes are the largest cell that we have in humanity. The size and shape is just really different from what you would expect for other cell types.

      Lauren: Right. So, in my previous example of turning a stem cell into a pancreatic cell, there might be morphological differences between those two cell types, but they aren’t necessarily at the scale that you would see in a stem cell to an oocyte. And, fundamentally, you aren’t changing the genome when you’re doing this transition from a stem cell to a pancreatic cell — the genome stays exactly the same. You’re just changing what genes are expressed. 

      When you’re creating an oocyte, you’re halving the number of chromosomes that you have in a cell. So, a mature oocyte only has half the number of chromosomes that a normal cell does, and that’s because it meets with a sperm that has half the number of chromosomes that a normal cell does. And now, when they fuse and fertilize, now you get the full complement. So, having to do both of those steps is far more challenging, and represents an additional hurdle that you have when creating these iPS or stem cell-derived oocytes. 

      Lauren: So, let’s talk about how you convert one cell type into another. What are, kind of, the — how do we think about how cell fate is controlled?

      Judy: So, the way that we think about cell types is often which transcription factors are present. And those transcription factors are proteins that will bind to the DNA and basically cause the production of RNA from a given genomic locus, so from a given gene in the DNA. So, transcription factors are used as the sort of master regulators of a cell type, where one transcription factor might be relevant. 

      Just to make it super simple, let’s say a certain transcription factor is relevant for neurons, and another one is relevant for hepatocytes. You would expect that all of the genes related to hepatocyte function and development are going to be controlled by that hepatocyte transcription factor. And in the neuron cell, that transcription factor is not present, so none of those relevant genes are made. That’s an oversimplification, but that’s kind of the idea of how these genetic regulatory networks work.

      Lauren: Right. And so, when we’re thinking about in the lab, in the clinic, if we want to convert one cell type into another, we can express these transcription factors, which then leads to the expression of all their downstream genes. And then that, kind of, reprograms the cell and says, “We’re a hepatocyte now, we’re a neuron now, we have this transcription factor that’s driving this — let’s call [it] the gene regulatory network.” And that guides the cell to a specific identity. 

      So, in this paper, what they’re doing is they’re trying to identify the transcription factors that govern this development of the oocyte so that they can take those transcription factors, express them in a stem cell, and then encourage that stem cell to become an oocyte. So, with that in mind, let’s start with how the authors identified these key genes — these transcription factors that are involved in oocyte development.

      The development of the oocyte

      Vineeta: My understanding is that they did whole transcriptome profiling of cells at different stages in mouse oocyte development. So, basically, compared the differential expression of lots and lots of genes at each of those different time points, and constructed a network analysis to nominate specifically not just genes, but regulatory genes. And they used a bioinformatic GO search to get to the subset of transcription factors that they believed were driving the evolution of the transcriptome through the differentiation process.

      Judy: Yeah. I think a lot of those genes were already known from studies that had done in vivo work, so I think it’s important to make that distinction. You can do this work in vivo, which means taking oocytes at different stages of development in a mouse and actually measuring the transcriptome there. Or, they created a sort of — an organoid model, for lack of a better word — where they put cells into an environment that is similar to what they would experience during oogenesis, and measured the transcriptome at different points in that process. 

      And so, I think the purpose of this was partially to find the genes that are relevant, but also partially to identify the period of time within this model system that should map onto the period of time in in vivo oogenesis. To be able to, basically, run their experiments, and know the right time period to look at the cells.

      Lauren: I think that’s a very good point. It was both a — can we identify the correct transcription factors? But can we also validate this very handy in vitro system that we can then use to do further study?

      Vineeta: And another way to think about it is — in biology, we talk a lot about necessity and sufficiency, and I actually think they did a better job with necessity than sufficiency. They proved that if you knock out any of these top-nominated transcription factors, that you can’t get differentiation past a certain stage. And you really need this set of factors to be expressed in order to get to a primary oocyte. Sufficiency is a much higher bar, right? You have to prove that the thing you got, the primary oocyte you got, can then go on to do all of the things that you expect it to do. And I would say it’s almost impossible to prove until you’ve generated the end state of, like, a mouse baby. So, they make some progress towards sufficiency, but less.

      Judy: There’s also a sort of curious result that I would like to, I don’t know — talk to the authors about and maybe understand a little bit more. Which is that they find these eight genes that they collectively called PPT 8, and they do all of their experiments with these eight genes. But there’s a paragraph in the paper that talks about how they found a subset of four that is sufficient to get the same phenotype you see. But if you have some subset of five, six, or seven that contains that subset of four, it doesn’t necessarily mean that that’s going to work. 

      So, there’s sort of this — they are sufficient, but then some other part of those eight — that set of eight — has some interaction with that set of four, such that they didn’t trust that the set of four was truly sufficient. So, there’s some complexity going on there that they’ve definitely moved on from by just using the set of eight.

      Lauren: Yeah. There does seem to be maybe a higher level of regulation that hasn’t been elucidated yet.

      Justin: A couple other things I thought were interesting, too, is that they had to also expose them to just some somatic cells from the ovary to get the development to take place. And so, I think it, again, reiterates the point that while these transcription factors are likely necessary, there’s also some dynamic signaling that’s happening from somatic cells that surround, as opposed to just being purely driven by those transcription factors. 

      One other thing that also caught my attention is the timeline that this all took place in. If you look at what’s happening in vivo, this is usually happening over seven or eight days inside the mouse model. But within the in vitro model, they saw this happen over a couple of days. And so, this is likely a necessary set of transcription factors and regulators, but there are likely other regulators — potentially checkpoints or others — that are absent in the system that’s allowing it to run through this process on a much more accelerated timeline. And potentially could explain some of the other issues that they see downstream with having the morphological appearance of an oocyte, but a lot of the functional aspects of it didn’t quite get that.

      Limitations of the current research

      Lauren: So, what elements of a functional oocyte were they able to recapitulate with these oocyte-like cells, and what’s missing?

      Judy: So, what they did successfully show was growth, the morphology. They also showed a couple of expressions of certain factors that we associate with oocytes, but what they didn’t show is that there’s the right dynamics surrounding the DNA. For example, meiosis didn’t occur, which — it’s also not something that they were aiming for, so I don’t think that’s the bar that we should hold them to, but there was no meiosis. So, what you end up with in the end is a cell with a lot of extra DNA in it. And then the other really important piece is that the methylation pattern is incorrect. It’s completely different from what you would expect in an oocyte.

      Lauren: The methylation — that’s one of the key epigenetic modifications. So, that governs how the chromosomes are packaged, and that leads to how accessible certain genes are to be turned off and on at particular rates.

      Vineeta: The maternal and paternal imprinting, the mechanism by which is most commonly methylation of the DNA, is really important for health and disease. We know now of many different disease states that are actually attributable to incorrect maternal or paternal imprinting. And so, it’s not a minor issue that methylation wasn’t solved, and one that we’d have to pay a lot more attention to as this research advances.

      Lauren: Yes, that’s a really good point. So, the real test — the final test of whether you got an oocyte or not — would be to fertilize it with a sperm and to grow a new being — in this case, a mouse pup — up from that. In this paper, they tried that, but it didn’t quite work. What was the result of this experiment?

      Justin: So, what they saw — and somewhat not surprisingly — is that when they did fertilize it, a very small percentage of the cells actually went on to cleave at all, and even of those that did, very few made it beyond the two or four-cell division. Which, a lot of the early cleavage in cells is dependent on having that haploid one single set of chromosomes. And so, in this scenario where they weren’t able to achieve meiosis, which means that they weren’t able to go in with a haploid cell, it’s unsurprising that when the cell was fertilized, it resulted ultimately in non-viable embryos — given that the chromosomal count mix is not consistent with traditional fertilization.

      The future of oocyte biology

      Lauren: Yeah. So, the paper makes some really great advances in our understanding of how an oocyte develops, what the gene networks are, what the transcription factors are that are regulating these — but there are still a lot of mysteries, and still a lot left to study in this process. When you think about these unanswered questions at the end of the paper, what are the questions that interest you? Where are you interested in seeing this work go next?

      Justin: I think for me, when I look at this paper, there are two, kind of, key avenues you could take. One is to look at this as, “Can we use the oocyte as a structural scaffold for other scientific applications?” And we see this happening already today, where enucleated oocytes, or rather the oocyte without the nucleus of the DNA material, are used in applications for mitochondrial disease to other potential therapeutic applications. Those are relatively limited. 

      And so, for me, the biggest question that this tees up is, “What are the next steps that need to be taken to really understand how we get the nuclear — how we get the meiosis portion of this correct? Because if we’re eventually going to reach, kind of, the vision that we outlined at the beginning — of having this be a critical asset and enabling greater access to fertility treatments — it’s really an absolute necessity and table stake in order for this to progress.

      Judy: Yeah, I totally agree with Justin. At the end, the authors say that this is a potential — a potential use case here is somatic cell nuclear transfer, which is another way of saying cloning. I think that there’s not that much need for this kind of a solution for those applications. I think we should really see this as a stepping stone toward [an] entirely ex vivo generation of an oocyte for the purpose of in vitro fertilization.

      Vineeta: Yeah. I think one of the things that’s hardest about this particular — the fertility use case of stem cell research is that, presumably, the parents of a prospective child want to see their genome represented in the progeny. A lot of other applications of stem cell research don’t actually have that requirement, especially if you can design creative ways to avoid immunosuppression and to create immune cloaking of a stem cell-derived therapy, and so on. You might actually envision in a lot of other fields an off-the-shelf stem cell-derived or iPSC-derived cell therapy that can be really therapeutic for a lot of patients with different diseases. 

      Here, we can’t have that, or at least it doesn’t solve some part of the core fertility challenge. And so, because you’re so dependent on actually running this process on a case-by-case basis with each couple, I think we just have an even further way to go on this. It’s not like you could create a bank at some point, and then differentiate them from that every time you need to spin up a new embryo. You really have to get the whole process right end-to-end from the point of a patient-specific iPSC cell line. And that’s really hard.

      Lauren: There’s not to be an allogenic option.

      Vineeta: Exactly. There’s no allo-embryo to be had here.

      Justin: And the bottom line for me is — in thinking of the ultimate translation of this, we’re obviously in the earliest of early innings in terms of this actually translating to being, kind of, that holy grail for fertility and the fertility treatments that we talked about earlier. I think this does clarify a lot of our understanding about what it takes to create a structurally similar cell to an oocyte. But ultimately, to reach the broader vision that we want, there’s still a lot of work that needs to be done.

      Judy: This is actually a really important step for developmental biology as well, because I think it’s one thing to do descriptive research, where you understand an existing system and you characterize all the pieces of it, and it’s an entirely different level of understanding when you can actually rebuild it. So, I think there’s the phrase Feynman says: “You don’t really understand something until you can build it,” or some variation on that. And so, this is a perfect example of a paper that’s using building to get to an understanding that is deeper than what we had before.

      Lauren: I think that’s a perfect note to end on. Justin, Judy, Vineeta, thank you for joining me on “Journal Club” today.

      Vineeta: Thank you, Lauren.

      Judy: Thanks, Lauren.

      Justin: Thanks, Lauren. Thanks, Judy. Thanks, Vineeta. This was fun.

      Lauren: And that’s it for “Journal Club” this week. If you enjoyed this episode, please subscribe, rate, and review wherever you listen to the podcast. And to learn more about how biology is technology, subscribe to our newsletter at a16z.com/newsletters.

      • Vineeta Agarwala MD, PhD is a general partner at a16z investing in bio and healthcare technology. She is also a practicing physician and adjunct clinical faculty member at Stanford.

      • Justin Larkin is a deal partner at a16z where he focuses on healthcare technology companies. Prior to joining the firm, he led strategy and operations at Verily and cofounded Wellsheet.

      • Judy Savitskaya is a deal partner at a16z where she focuses on bio companies. Previously, she worked on synthetic biology research at UC Berkeley and was a computational modeling and neuronal networks researcher.

      • Lauren Richardson

      Value Versus Volume (in Healthcare)

      Todd Park, Vijay Pande, and Hanne Winarsky

      The way we pay for healthcare in the US has long been by fee-for-service: per doctor visit, per test, per surgery, per hospital stay. But that system has led to rapidly escalating volumes of services and cost to the system—without actually improving outcomes. What if we shifted everything towards paying for value—and outcomes—instead? In this episode, Todd Park, co-founder and executive chairman of Devoted Health, and formerly Chief Technology Officer and technology advisor for President Barack Obama; a16z General Partner Vijay Pande; and Bio Eats World host Hanne Winarsky—talk all about the megatrend of value-based care, and how it is redefining healthcare itself. Why is now the moment for this massive shift? How do we implement it? What does it mean for doctors and patients, insurers and policymakers? What is tech’s role in making it possible, and what’s the business model and incentive for creating value?

      Show Notes

      • Discussion of what value-based care is [1:20], and how it emerged as an alternative to the current system [4:35]
      • The importance of targeting healthcare [7:11], and how to scale the value-based system [13:11]
      • Why there is no “silver bullet” to fix the current system [16:00], but how we might shift toward a new paradigm [24:13]
      • Discussion of first steps that may be taken [31:06] and what it would look like to rebuild the healthcare system from scratch [34:00]


      Lauren: Hi, I’m Lauren.

      Hanne: And I’m Hanne. And this is “Bio Eats World,” our show where we talk about all the ways that our ability to engineer biology and reengineer healthcare is transforming the future. And when it comes to reengineering healthcare, there’s one concept that gets a whole lot of airtime — the concept of value-based care.

      Lauren: Value-based care is a term that we’ve thrown around on many different episodes about how the healthcare system is evolving, but we’ve never really gone straight to the heart of the matter.

      Hanne: So, that’s what we do in this episode with Todd Park, co-founder and executive chairman of Devoted Health, and formerly chief technology officer and tech advisor for President Barack Obama — along with a16z general partner, Vijay Pandey, and me, Hanne. So, what exactly is this big mega trend of value-based care all about? And how is it redefining what we think of as medicine, treatments and healthcare? What does it mean for doctors and patients, insurers, and policymakers? Why is now the moment for this big shift, and what exactly is tech’s role in it?

      Defining value-based care

      Hanne: We hear the term value-based care thrown around an awful lot, but we’ve never really talked about what that means. So, this conversation is really about what is value-based care? How do we implement it? Why is it better? And what is technology’s role in that? So, maybe we could just start with — how is it different from the healthcare system today…

      Todd: Oh, it’s really different from how the healthcare system generally works today, famously, or infamously.

      Vijay: Even aspirationally. I mean, when you think about how the plumbing works.

      Todd: That’s right. The root cause, honestly, is how the healthcare system is paid for. So, historically, U.S. healthcare has been paid for in what’s called, “a fee-for-service.” Frank, right. So, doctors, hospitals, and healthcare providers are paid per thing they do — like, per doctor visit, per test, per surgery, per hospital stay. That has led to a situation where we have rapidly escalating volumes of services being delivered. But, unfortunately, we don’t actually have commensurately improving outcomes. We spend the most per capita of any country in the world. We rank at the bottom of the developed world on metrics like avoidable death, preventable death, adverse events, healthy life expectancy — and that’s because we fundamentally have a pay-for-volume payment system. 

      If you want to change the situation, then change how healthcare is paid for. Basically, [we should] move away from pay for pure volume to engage in value-based payment — which, in a nutshell, is a payment system that actually financially supports, aids, and abets right care, right place, right time. Value-based care, to me, is the right care, including non-clinical support, delivered in a consistent, coordinated, and proactive way that both improves outcomes and lowers costs, [thus] saving money.

      Vijay: It’s interesting to think about how this came about historically. Basically, [it was] a perk given to workers to help them stay with a given employer instead of somebody else. And so, that perk was, like, “We’ll get together, and we’ll pay for your medical bills or pay for things.” It’s kind of akin to, like, I don’t know, like if we gave a perk to say, “We’re going to pay for your plumber bills,” or, “If you have some major catastrophic problem, we’ll pay for the plumbing.” But that doesn’t mean we’re going to keep track of your house, or try to see if your plumbing is in good shape to avoid the problems. We’re not paying for that. That was never part of the plan.

      Hanne: We’re not paying for copper pipes. We’re paying for when something really goes wrong.

      Vijay: We’re paying for when the pipes burst. But we’re not paying for maintaining your house. That’s something that was always assumed to be on the patient side, so to speak. The thing is, we’ve gotten very good at trying to come up with therapies for cancer, for stroke, for massive heart disease. But, what we’ve come to realize is that that’s actually more expensive, because if we wait that long, those therapies can work. But [these therapies are] painful in many different ways, emotionally and physically for the patient. We want to get there before the pipes burst. Think about the house and the overall health of it.

      Hanne: So, can we talk about when this concept started to emerge? Why was there this gradual dawning of realization that [value-based care] was a better north star to orient towards, or would everyone say this is a better north star?

      Todd: I think in the last five years, the move to value-based payment and care has gone from, “Is it going to happen?” to, “It’s going to happen. It’s just a question of how fast.” It’s an idea that’s been around for decades. I mean, it goes in the category of a super obvious idea.

      Hanne: Right. Right.

      Todd: For example, if you have diabetes, or hypertension, or congestive heart failure, there’s an incredibly well-known best practice pathway in the form of medication regimes that get adjusted, based on each patient’s evolving situation. Along with very basic coaching on diet and activity. It’s very, very straightforward to execute. If I’m a primary care doctor being paid fee-for-service, and I’ve got a patient in front of me who has diabetes and hypertension, and I want to spend an hour with that person — to really help educate them about their condition, and really get into it, in terms of coaching and what to do, in terms of coordinating their care and providing them with the right support — I literally can’t afford to do it. Because I have to, in that same hour, see another three patients and get paid the fee per those services to stay alive. It is a bankrupting action for me to actually spend the extra hour; it is far more difficult to actually financially support and execute those pathways than it should be.

      Poorly controlled chronic illness is the greatest single driver of more serious events like heart attacks, strokes, kidney disease, eye and nerve damage, and vascular disease. And it’s just nuts. We as a country don’t do these incredibly basic things to keep people healthy during these chronic conditions. In addition to these patients being cared for the way they should be, you will save so much money, because I’m going to actually save you one, two, three hospitalizations that cost $20,000 each. We’ve invested so much in developing incredibly advanced therapies for acute situations, but we aren’t doing the basics. Because systemic execution of the basics, with process control and improvement that every other industry would find routine — it really does require value-based payment and value-based care as the paradigm to make that happen.

      Vijay: Yeah, otherwise, there’s no incentive.

      Todd. Right.

      Getting patients the care they need

      Hanne: Todd, when you say the right care at the right place and the right time, can you talk about what exactly that means? And what would that mean on the entire country model?

      Todd: Maybe the simplest way to explain it is, as opposed to me as a primary care doctor being paid X dollars to see people for 15 minutes, I am given what’s, in effect, a global budget for all medical spending — physician, drugs, tests, surgeries, and hospital stays — for my patients. It’s risk adjusted. If I’ve got a patient panel that has significantly sicker patients, then my budget’s higher, because essentially, the budget’s set to be equal to what the healthcare system has generally been spending to care for folks in a situation where patients have not been getting — in a systematic, universal way — right care, right place, right time. Highly prevention-oriented care. So, I have this budget that I’m working off of, and my goal as a primary doctor is to proactively get patients the right care in the right place at the right time. If you do that, then it’s been shown, for example, that you can cut hospitalizations versus the status quo by 40% or more.

      Hanne: Incredible.

      Todd: It’s a significantly positive financial ROI transaction for them, because the way these arrangements work is that they get a share of the savings from keeping folks out of the hospital. That’s why primary doctors are so much happier when you put them in value-based paying arrangements, because under those arrangements, they can afford to spend that extra time with the patient. They can afford to hire personnel on their teams to help care for that patient, and give them the really straightforward care and support that those patients need to actually stay healthy [and] out of the hospital. 

      The primary care doctor, basically, generally speaking, doubles their pay under value-based payment by delivering care the way they always thought they were going to [when] they graduated from med school. As a bonus, then you significantly increase your pay, because you’re saving the healthcare system so much money.

      Vijay: Yeah. The proof of this is where you have a cohort at risk, and it’s the same provider, and the outcomes are fundamentally different.

      Todd: Oh, yeah. And the evidence is conclusive, as well as it being commonsensical.

      Hanne: It strikes me that — in the category of super, super obvious, but it’s a different way of measuring. It takes a different amount of time, a different perspective. So, how do you overcome that challenge, to measure how long somebody stays healthy for?

      Todd: One interesting challenge that primary care doctors face is the so-called “foot in two boats challenge.” It’s part of the exercise of transmogrifying from a fee-for-service paid operation to a value-based paid operation. Fundamentally, they’re two completely different kinds of operations. So, in a fee-for-service paid operation, if you’re a primary care doctor, you have to maximize throughput in order to actually stay alive. In a value-based payment paid operation, you’re focused on how do I take the best possible proactive care of my patients and keep them out of harm’s way.

      Hanne: Prevention.

      Todd: Exactly. It’s much more proactive, personalized care of people, and following up with them out of the office to ensure that they have their medications. Any changes in their circumstances actually get reflected in a change of their treatment. That things are going well, and they have the right support, and the right iterations are made to their care.

      Hanne: So, it’s a different muscle.

      Todd: Yeah, it’s a completely different mode of operation. And so, the “foot in two boats” is a famous articulation of this problem. What if part of my patients are being paid for fee-for-service, and part of my patient panel is being paid in a value-based payment mode? Then, I’m trying to do max throughput and also proactive, systematic care.

      Hanne: Yeah, and you’re stretched super thin.

      Todd: And you’re trying to be, like, two different modes at once. That’s why the most successful provider organizations under value-based payment have gone all in on one boat. And that’s helped by the fact that, for example, Kaiser and CareMore have a built-in health insurance plan, inside themselves, that then actually pays — in a value-based way — the providers that they employ, that operate in a value-based way.

      Scaling value-based care

      Hanne: So, is that part of the reason why we haven’t seen it happen faster? Because you have to create something from the ground up that is a total systemic shift?

      Todd: I think that’s a really, really good way of encapsulating why there hasn’t been this kind of shift nationally — because you have to go all in. As you think about the Kaisers, the CareMores, and even the ChenMeds — they take global risk payments from health plans, meaning, they go to health plans and say, “Pay me a global capitation payment, in effect, for all care expenses.” So they, in fact, are their own kind of mini-payer, if you will. Their existence proves that if you actually do that, and you get people the right care in the right place at the right time, it both leads to significant improvement outcomes and lower costs.

      The common denominator across the successful early American experiments is that they’re a full payer/payer provider stack who can therefore actually act with the right incentives. And, by the way, have the right information at the fingertips to then take the right care of people in the right place at the right time. That is a really tall order to replicate. The rest of America is neck deep in fee-for-service, and the payers pay a fee-for-service. The providers operate fee-for-service, and all of their business systems and operations are optimized for fee-for-service.

      Hanne: Yeah, it sounds like trying to renovate the foundation of a house, almost.

      Todd: Right. You’ve just got to honestly roll in with a new house.

      Hanne: Right. Build one.

      Todd: That’s right. I mean, that’s effectively the magnitude of the challenge. That’s a good way to think about it. And so, the question has been, “How do we, as a country, take those archetypes — take those results — and scale them to much more of the country — to the whole country?” As opposed to it being available and accessible to limited populations of people in certain pockets of the country.

      Vijay: The thing Todd really pointed directly to is that we have a sense for what to do, but how do you scale it? You have all this data and all this logistics. It’s just having to make tons of different decisions in complicated ways. This, ironically, seems like something that’s very well suited for tech, something where if you could build the infrastructure to do that, you could take all the little things that have to be done and do them that much more efficiently. Think about something like Amazon. Amazon and Sears, they both sell things. And actually, they both sold things over the internet. But, by taking a tech-first approach all the way through, whether we’re talking about the website, or the back end, or delivery, just everything — that’s the best bet to try to save and improve every little part. Because there’s not going to be a silver bullet that, like — with this one idea or killer algorithm, that suddenly healthcare is solved, or healthcare is easy or cheap. It’s going to be lots, and lots, and lots, and lots, of little things.

      Hanne: When you say tech from A to Z, and there’s no one silver bullet, but it’s all little aspects and incremental accelerations or improvements, what exactly do you mean? Where does it look different?

      Todd: You have to think about it as honestly reinventing each layer of the stack of American healthcare. You want to have a health insurance plan layer that’s explicitly optimized, from birth, to support value-based payment of care. So, the historical American health insurance company was born in a world, as Vijay said, where they’re paying fee-for-service bills. And so, that’s what they did. As those bills began piling up, escalating speed, their response was, “Okay, I’m going to erect administrative infrastructure that micromanages doctors and patients, and polices what they’re able to do, by making them ask me and give permission.” That’s probably not how they pitched it to people, but that’s effectively what it was.

      Hanne: Yeah.

      Vijay: Because the one thing they can do is say no.

      Todd: Right. So utilization management and pre-authorization, right? And, things that, for all the physicians listening to this, are epithets. That, then, led to doctors erecting their own administrative infrastructure to interact with the payers, to make arguments about what should actually be paid for, which has massively escalated administrative spending on healthcare in America. It also led to escalating mistrust, distrust, between patients and insurers, and doctors and insurers. It has generally not solved the problem. You want a payer that says, “Look, we’re not going to pay a fee-for-service, we’re going to actually pay in a value-based way,” which is a totally different mode of payment.

      Hanne: But that feels like it’s about, again, a kind of framework and mindset shift and payment shift. I don’t understand tech’s element in that.

      Todd: You cannot do anything I just said without software, and you certainly can’t do it scalably without software. So, first of all, the notion of paying a primary care doctor in a value-based way — you’ve got to be able to, for that patient panel, for that doctor, set the right risk adjusted global budget. You’ve got to actually be able to actually track everything that actually happens. So you’ve given the doctor visibility into what’s happening to their patient base. And you’ve got to provide the doctor with data from all points of the compass — pharmacy data, lab data, electronic medical record data, claims data about what kind of care that patient’s getting from across the system. To put all that at the doctor’s fingertips, to actually make sure this person is healthy. Beyond that, you need to establish a relationship with a member that is incredibly supportive, where you’re also getting the member information about where they stand and what needs to happen, before they even know it.

      Hanne: So, there’s like a whole other information flow happening.

      Todd: That’s right. Mark Smith, who’s a visionary healthcare leader, has this great analogy. He says, “I go to Harrah’s casino. And before I even know I’m thirsty, there’s someone there with a drink. Just when I’m about to leave the blackjack table, someone says, ‘Can we comp you some free food? Take some chips while you’re at it.’”

      Hanne: As if by magic, yeah.

      Todd: Exactly, right. And so, he says, in all seriousness, healthcare needs to be like that. So, before you even realize that you’re about to have a problem, someone calls you and says, “Hey, I think you might want to actually get this med. I think I want to see you and check something out.” If the American health system operated like Harrah’s casino, it would save many more lives and cost a lot less. 

      This is a classic data and tech problem. I have friends who work in AI in healthcare, and they said, “Look, at this point, the holes in American healthcare are so big you can see them from space.” So, if someone hasn’t refilled their med — they have a chronic illness, they don’t take their meds, they’re going to the hospital. And so, the low hanging fruit in U.S. healthcare is so plentiful — it’s fruit pies on the ground, right, with a cold glass of milk next to it. There’s so much progress we can make if we apply tech-enabled process control and improvement, as has been routine in virtually every other industry, to healthcare. But to do that, you need to actually have a full stack, payer provider ecosystem, where there is a business case for the use of those approaches.

      Vijay: All of the innovation we’ve been thinking about has been — let’s say, in therapeutics or how medicine is done — what we’re really talking about is basically how to get better at the existing game. If the game is fee-for-service — how to do lots of services, better services, more services. Almost like if you’re building a machine to do chess, you can do chess really fast, but this is about getting rid of the chessboard entirely and playing a different game. The real point is in giving us the whole stack. That’s really, really hard to do. 

      But to your point, I think it is particularly intriguing to ask now, if we’re going to start with a new game, how can we set up the game to have the best chance to win — the best chance to benefit the patients, and to reduce costs? How do you construct that game? How do we make sure that we’re constructing the right game? Because whatever game we do, someone’s going to try to win, and will win at those rules. But that may not be what’s best for patients and may not be what’s best for cost.

      Hanne: It’s interesting, because when you’re describing this, I’m thinking about all the ways that healthcare as an industry is particularly challenging to introduce innovation into, because of how unique it is, and certain regulatory hurdles. And I’m also thinking about the move towards more consumer facing, and the market-driven forces that are pushing in that direction. So, I’m going to ask either a really hard or really dumb question. I don’t know which it is, but — how does this value-based shift work with that, kind of, market-driven shift? You know, you see these two big forces. Are they opposing tides? Do they come together? What’s the fit between those two?

      Todd: So, in a nutshell, look — if you lower costs and improve outcomes, you can actually put a better health plan product in front of people in a market like the Medicare Advantage market. And more people will buy it.

      Hanne: So it’s quality really?

      Todd: It’s quality, and it’s cost savings. It’s then the ability to use those cost savings to fund better benefits in your health plan than the competition. And this intersects with consumerization, right. So, in a market where you can offer a health plan that is better for consumers, there’s also a significant positive ROI in investing in and delivering on a world class consumer experience. Why is this? Because if you actually deliver a world class consumer experience, that’s both helpful to you as you seek to win more customers, but it’s also directly helpful to your ability to deliver better outcomes and lower cost. Why? Because if you’re actually working with a member, and they do not trust you, then when you ping them and say, “I think you…”

      Hanne: It all breaks down.

      Todd: Yeah, the member will say, “Pound sand. I don’t trust you further than I can throw you.”

      Hanne: Yeah. So, it’s about strengthening the relationship so that when you need to direct them, it works.

      Todd: Exactly. Both when you need to direct them, they will actually listen to you, and B, when they have a problem, they’ll call you.

      Hanne: Oh, wow.

      Todd: As Atul Gawande says, “The true superpower of a primary care doctor is that people will tell them things that they don’t tell anyone else.”

      Hanne: Oh, my gosh.

      Todd: So, basically, a patient will call a primary care doctor and say, “I am not feeling so great about X,” right? And that’s an early warning signal to jump on. So, if you are a health plan, and you are genuinely trusted by your members, not only will they listen to you when you call them and suggest something — or ping them and suggest something…

      Hanne: They tell you more important stuff.

      Todd: They will call and say, “You know, I’m having this issue. I’m not sure what’s going on. Can you help me?”

      How to shift towards value-based care

      Hanne: Yeah. So, if this shift towards this model depends on gathering new types of data and knitting them into a more holistic picture of the patient for the provider and for the whole healthcare system, what are the types of information that we’re using now that we haven’t been using before? How are we thinking about that whole picture of the patient in a different way? And what is it that we need from the provider lens for this whole model to work?

      Todd: Yes. So, building on what we were talking about earlier about the operation of a primary care physician practice, you’re really moving from a paradigm where you’re maximizing throughput to  a paradigm where you are maximizing the outcomes you’re delivering to patients. And so, armed with the right comprehensive data, you want to actually make sure that you are getting members the right care in the right place at the right time in a highly coordinated, proactive way. The role that technology plays in that is not just the assembly of data and the catalyzing of the right action, but also actually enabling virtual care and home care on an epic scale. Because it’s increasingly obvious that the right place for the right care is the home.

      Hanne: Right. Not getting you into the ER, not in a hospital.

      Todd: Or forcing you to come to the medical office, right. So, going to the early examples of someone who has diabetes, or hypertension, or congestive heart failure, as opposed to you going into the medical office, which is a big logistical exercise. Getting a measurement taken, and then the doctor deciding to put you on this particular med and saying, “Come back in three months.” You come back in three months, they make another adjustment, and then so on and so forth. That takes months or years to get your chronic condition under control. 

      Instead, a patient has a continuous glucose monitor that streams data in the software. It has a wireless scale — streaming data into software. It has a wireless blood pressure cuff — streaming data in the software, which then analyzes the data in combination with humans, if necessary — then triggers a set of actions where the care provider can, basically, through a televisit, say, “Okay, I’m going to adjust your med by X,” and then see in 24 hours what the result was. And then adjust it again, then adjust it again, and adjust it again. Within a matter of one to two weeks, literally get that chronic condition under control — vis-a-vis a pattern in the old world, where it would take years, and in a way that’s dramatically more convenient for the patient. You’re basically dramatically — through remote monitoring and virtual visits aided by software, you’re both detecting issues a lot earlier, and you are increasing, by orders of magnitude, the speed of “intervene, see what happens, and adjust.”

      Hanne: The whole feedback loop becomes much faster.

      Todd: Exactly. That’s one of the many possible examples. But overall, that’s what’s happening. The accessibility of care dramatically improves, the richness and the timeliness of information dramatically improves, the frequency of touch points dramatically improves. And you experience a significant acceleration of improvement of outcomes, and associated with that, lowering of cost.

      Hanne: So, I’m thinking about providers listening to this and thinking, “Well, the information itself sounds like a dream, to have all that at your fingertips, to understand all that about your patients. But actually parsing all of that?” How do we make sure that we deliver that to providers in a way that it’s not this giant, hairball mess of more data and information that they then need another administrative layer just to figure out?

      Todd: Well, this is a classic problem where software could help — is helping today. What I would say is that the optimal approach to acting on this opportunity is where you’re fusing software and humans together in an optimal combination, such that you could actually deliver the actual result.

      Hanne: Right. So, it’s precisely where those two meet that you have to make sure it’s joining well.

      Todd: Yeah. You want to design a tech-enabled service, which leverages both software and humans in the right combination with software doing a ton of the work to be able to efficiently, effectively and scalably deliver the actual outcome to the patient.

      Hanne: So, if we’re talking about this massive shift from treating sick people — treating [the] chronically ill — to keeping people healthy, healthier. Preventing illness, catching it earlier — how do we begin to shift the whole system conceptually around that? What other types of information do we need to be thinking about? Transportation, food security? Or new types of treatments that we haven’t been thinking about?

      Todd: If you have a full “payvider” stack — and by that I mean a payer and provider healthcare stack that is optimized for value-based payment and care. One inevitable additional layer of action you take on is non-clinical drivers of terrible health outcomes. And again, you talk to every primary care doctor in America, they say, “A huge portion of what actually drives terrible health outcomes for my patients are not clinical.” So, social determinants like transportation, food, how your house is equipped, social and emotional support, etc. I’ve talked to primary doctors in Florida who say, you know what the number one thing that you could do to help people with COPD in Florida, and reduce adverse clinical events?

      Hanne: No. What?

      Todd: Get them air conditioning.

      Hanne: No.

      Todd: One of the things — it makes a lot of sense for payvider stacks to provide aid to members that includes fixing up your bathroom, with bars and mats that stop you from slipping and break your hip.

      Hanne: Oh, my gosh, yeah.

      Todd: To be able to actually get you healthy food that is tailored for your chronic condition, and other non-clinical artifacts and services that have a huge impact on ultimate clinical status and health status.

      Hanne: It’s so interesting, because while you’re telling that story, I’m thinking about the unseen caregivers of those patients that are doing those things now, and how if the healthcare system can take over that role of the son, or the daughter, whoever it is taking care of that person that needs that extra air conditioner, or needs the handrails set up. It feels like the healthcare system is doing more, but it’s actually making it more cost efficient. It feels like everybody wins then.

      Todd: I think you hit upon something very important there. In that, for the average patient, the American healthcare system can seem very forbidding, and confusing, and fragmented. And it’s very hard to understand. It’s the healthcare system that only works when it works, because the individual doctors and nurses in it are by far the best in human history. The problem with our healthcare system isn’t the doctors and the nurses. Far from it. It’s that the system in which these professionals are operating is so disorganized, fragmented, confusing, incentive misaligned, information poor, reactive. The notion of having a professional daughter and son to help you navigate the system as a patient, and be able to get the right care, right place, right time — is, I think, a crucial, crucial role. That’s, again, another role where technology can be extraordinarily helpful in helping that happen. So, a major barrier for a lot of patients is literally transportation. This ability to literally get to your doctor’s appointment, or get to where you’ve got to go as you travel through the healthcare system. And so, a very significant benefit to patients getting the right care, right place, right time is to actually provide them with medical transportation.

      Hanne: Right. It’s not just the doctor’s visit, it’s getting to and from the doctor’s visit.

      Todd: It’s literally getting to and from the doctor. Key. Really important, completely solvable problem in a tech-enabled way.

      Vijay: You know, the thing about social determinants is that I think there’s so many more low lying fruit. It’s not something where people have put much effort. It’s not something that payers have really thought about. 

      I think there’s a huge opportunity for just actually applying analytics, data science, tech, to figuring out what these low lying fruit areas are, and what could be done about it. It’s probably not even a hell of a lot of dollars, it’s probably just figuring out what’s the best place and what’s the biggest need, and when is that need. And that combination of who, what, where, when — that’s particularly hard to figure out. If we could provide a coach or a parent, or, like, a doctor that was with you 24/7, that person would probably know, but we can’t know that. So, that is where tech can sort of fill in the gaps.

      A new future for healthcare

      Hanne: So, just to go back to where we began, and think a little bit about this major new orientation towards, like — it’s a very obvious shift, but it’s a massive shift to make for the system, right? When the entire architecture of the system and the way it’s been developed over the last decades, and the mindsets and the education and the processes — all those things have to be shifted. If you were able to full stack the entire system — if you were able to start from scratch — what would that look like today? What does a real revolution American healthcare look like?

      Vijay: I would love to just go right at the question, which is, what would it take to make the American healthcare system the envy of the world?

      Todd: I think a way that the U.S. can leapfrog, going from the bottom of the rankings in the developed world to the top, is to make an increasingly strong move toward value-based payment care. See the rise of more and more tech-enabled payvider stacks across the country. Have those tech-enabled payvider stacks, in all their different forms, compete with increasing energy on the basis of outcomes and cost and consumer experience.

      Vijay: And fueling innovation that way.

      Todd: And fueling massive innovation, right, versus if the entire U.S. health system were actually being run centrally. We just have to get to a place where we create the right magnetic field from an incentive standpoint, by continuing to move strongly toward value-based payment. So that value-based care innovation and value delivery innovation can really blossom through these tech-enabled payvider stacks and all their configurations competing with each other.

      Vijay: Well, how can that come to be? I think that makes a lot of sense from, sort of, a bottom up, but how do we get to it from the top down?

      Todd: I actually think that in the Medicare space, all the conditions already exist to enable this to happen. It is entirely possible to build tech-enabled payvider stacks that are wired for value-based payment and care with no incremental policy change. One key to that is that in the Medicare space — say the average Medicare Advantage plan keeps a member for eight years. The cycle time between intervening and helping and saving money is short, given the population. And so, that means that if you’re a tech-enabled payvider, or tech-enabled payvider stack in the Medicare space, you have a very strong business case to make investments in better care. And you’ll realize the payoff. One challenge in value-based care oriented payvider stacks in the under-65 segment of the population is that people tend to have their insurance plan for far shorter than eight years. And so…

      Hanne: And that’s because you change jobs, you change…

      Todd: People change jobs.

      Hanne: …situations. Yeah.

      Todd: They go from Medicaid, to the exchange, to an employee and back. And the cycle times between intervention and payoff, on average, tend to be longer, because you are dealing with a population that doesn’t have the level of illness burden of seniors. So, I think a really interesting area of policy innovation could be how would you actually solve that problem? Because if you did, then I think that it would then create the right kind of business case support to do the tech-enabled payvider play outside the Medicare space. I think it’s still possible to do, it’s just a lot harder for this reason.

      Hanne: And to your point about the importance of the relationship with the patient being such a powerful tool when you only have a short-term relationship.

      Todd: That’s right.

      Vijay: You know, what we’ve been talking about trying to innovate a complicated system of analytics, decision making, logistics. These are all things that are, in many other areas, well approached by tech, well improved by tech. If you think about even just the A/B testing for websites, or for services, or for anything — that just constantly, sort of, trying new things, experimenting, having the analytics, seeing if it improves, and carry on. Ironically, that’s also not alien to medicine, that’s also an RCT, in a sense. I think we need to innovate, we need to try new things, see what works, see how it changes outcomes, and incent people to do it. Once you realize that you can set up a system where we are incented for innovation, that will automatically bring tech in.

      Hanne: A system that’s incented for value, that’s incented for innovation, and that’s incented for actual health before we even get sick.

      Thanks so much for joining us on “Bio Eats World.” If you’d like to hear more about all the ways biology is technology please go subscribe to the a16z bio newsletter at a16z.com/newsletter, and of course, subscribe to “Bio Eats World” anywhere you listen to podcasts.

      • Todd Park

      • Vijay Pande is a general partner at a16z where he invests in biopharma and healthcare. Prior, he was a distinguished professor at Stanford. He is also the founder of Folding@Home Distributed Computing Project.

      • Hanne Winarsky

      Amazon Narratives — Memos, Working Backwards From Release, More

      Colin Bryar, Bill Carr, and Sonal Chokshi

      When you hear stories about Amazon’s “invention machine” — which led to a company with not just one or two products but several successful diverse lines of business — we often hear about things like: Memos, six pages exactly and no powerpoints at all!; or, the idea of just “work backwards from the press release”; and other such “best practices”… But what’s often lost in hearing about these is the context and the details behind them — the what, the how (as well as their origin stories) — not to mention how they all fit together. Knowing this can give us insight into  how all companies and leaders, not just Amazon and Bezos, can define their cultures and ways especially as they scale. After all, Amazon was once a small startup, too.

      So in this a16z Podcast with Sonal Chokshi — the very first podcast for the new book Working Backwards: Insights, Stories, and Secrets from Inside Amazon (out February 9) — authors Colin Bryar and Bill Carr share not only how Amazon did it, but how other companies can do it, too, drawing on their combined 27 years of firsthand observations and experiences from being in “the room” where it happens. Bill was vice president of digital media, founded and led Amazon Music, Amazon Video, Amazon Studios; and Colin started out in the software group, was a technical vice president, and then, notably, was one of Jeff Bezos’ earliest shadows — the shadow before him was in fact Andy Jassy, president and CEO of Amazon Web Services (soon to be CEO of Amazon).

      The two share not only the early inside stories behind (ultimately) big business moves like AWS, Kindle, Prime — but more importantly, the leadership principles, decision making practices, AND operational processes that got them there. Because “working backwards” is much, much more than being obsessed with your customers, or having company values like “are right a lot”, “insist on the highest standards”, “think big”, “bias for action”, and more. The discussion also touches on hot-topic debates like to lean-MVP-or-not-to-be; the internal API economy; do you even need a chief product officer; and if you need less, not more, coordination as you grow. Can startups really be like Amazon? Yes: and it comes down to how leaders, organizations, and people at all levels decide, build, invent… using the power of narratives and more.

      Show Notes

      • The early days of Amazon and the development of the company’s 14 principles [3:18], including one that says leaders should be “right a lot” [6:53]
      • Amazon’s use of written narratives over PowerPoints [9:05], and how meetings are conducted using narratives [15:55]
      • The tenets that guide Amazon’s decision-making [24:00]
      • Working backwards from a press release [26:28], using AWS as a case study [34:56]
      • Discussion of lean startup principles [37:31], and how Amazon’s core principles balance each other [46:21]
      • Advice for startups, including reducing coordination and centralization [51:16]
      • Lessons learned from shadowing Jeff Bezos [58:50]


      Sonal: Hi, everyone. Welcome to the a16z Podcast. I’m Sonal, and today I have another one of our special, exclusive first-looks-at-a new-book episode — and it is both a very timely and evergreen topic, because the new book, coming out this week, is titled “Working Backwards: Insights, Stories, and Secrets from Inside Amazon.”

      In it, authors Colin Bryar and Bill Carr — who between them have a combined 27 years of experience in the company — where Bill was vice president of Digital Media, founded and led Amazon Music, Amazon Video, Amazon Studios for a decade. And where Colin started out in the software group, was a technical vice president, and then notably, was one of Jeff Bezos’ earliest shadows, a legendary program there. Fun fact: the first shadow before that, I believe, was Andy Jassy, president and CEO of Amazon Web Services (and now to be CEO of all of Amazon). The book actually shares the origin story of AWS, among other businesses there, which we touch on briefly — though, as a reminder, none of the following is investment advice. Be sure to see a16z.com/disclosures for more information.

      But in any case, our focus today is really on what is the Amazon way — and can other companies really adopt certain best practices, too? In fact, as fast-growing companies establish and find their way, how do they define and scale their culture, processes, and more? We actually spend most of the episode digging, in detail, into two operational practices in particular — the infamous memos-instead-of-PowerPoints, and working backwards from a press release and FAQs. Given the presence of those two topics in tech folklore, and lots of misunderstandings as well — so I actually probe for the origin stories, the specific details of how they do and don’t work, and other nuances so organizations of all kinds can take what they want or need.

      Finally, we also debate within this episode the tradeoffs of lean and minimum vs. maximum viable products (and whether the emphasis is on the wrong letter there); whether product managers have a role in companies organized like this; and more topics throughout.

      Amazon’s early days

      But we start by very briefly discussing the foundational principles — and actually, the first question I want to start with, especially since I hate the “why’d you write this book” question) — Colin, Bill, honestly, there’s a tendency for folks telling these stories, these kind of narratives, to do a sort of hindsight is 20/20, not accounting for attrition data or the failure cases as well. So, part of me is skeptical that startups can do what Amazon did. And, what’s most notable, too, is that Amazon had not just one or two or three product lines, but literally entirely different yet successful lines of business under one roof. So, what drives that? And can startups really relate to the Amazon story then?

      Colin: Well, one thing is that the businesses you mentioned, they are substantially different. They require different expertise, they’re different customer sets — AWS is B2B, there’s streaming video, there’s the e-commerce business — but they all have one thing in common, and that’s what we talk about in the book, and we call it The Invention Machine. Which was the process and principles that Amazon used to develop these businesses. And the ones that we talk about, they started off as ideas on a whiteboard or emails — which many people were skeptical we should even do. Some of them grew into household names, but they all started off very small, and with just one or two people.

      Bill: I would first start by going back in time a little bit, and place you sort of where Jeff Bezos was and where all entrepreneurs start out. So, at the beginning, Jeff worked out of a simple office building with a handful of employees, and he would be hands-on for everything. The very first customer support emails, like, Jeff wrote or co-wrote. He could review the work and think about all the policies. He could direct the team, and set the tone and the pace.

      Well, that works just fine when you’re an early-stage company and you know there’s fewer than 20 of you, and you can all do a stand-up each morning, and you can be hands-on. But guess what? That completely breaks once you start to grow like a weed, and you found product-market fit, and suddenly you’ve looked around and realize you’ve got a team of 130, 200, 500 — and, you realize that there’s all kinds of decisions and meetings happening around you. You can’t be part of every decision. And so, to me what’s most remarkable and notable, is that Jeff sought to figure out ways to actually inject his lens of thinking into all those meetings — and then sought to create a bunch of processes that would reinforce the way he would think about the work, or do the work itself.

      Sonal: It’s this idea of operationalizing ‘the invention machine,’ as you guys are describing it. Some of those principles and processes — at a high level, to quickly summarize, to set context for our listeners — they range from customer obsession, ownership, invent, and simplify, “are right a lot” for leaders, learn and be curious, hire and develop the best, insist on the highest standards, think big, bias for action, frugality, earn trust, dive deep, have backbone, disagree and commit, deliver results — which, I love. And we don’t have to unpack all of those, because I will take, like, all day, and it’s the whole reason your whole book exists.

      I do want to ask about one before we go into some of the other practices, which is around leading. And the one that really intrigued me was #4, “are right a lot.” And you basically write that, “Leaders are right a lot. They have strong judgment and good instincts. They seek diverse perspectives and work to disconfirm their beliefs.” And I love this, because I always think to myself, “Yeah, dammit, a leader should be right all the time. Their instinct should be damn good.” Tell me more about that one, because that one made me chuckle a bit.

      Bill: Yeah this is — and just to be clear of course, we did not write that. That is Amazon’s words that Jeff and the S-team, being his direct reports, painstakingly reviewed to come up with that exact language to define that principle — as with all 14 others. They sweated over the details of every word, every sentence.

      And, this principle, “right a lot” — in some ways, it’s very straightforward, which is that as you move up, an early-stage CEO as they grow and progress, they need to be more in the mode of delegating important work and auditing work, but their most important job frankly is to make decisions. And, many decisions will come to you — whether that’s presented with a spreadsheet, a document, PowerPoint. There’s all kinds of data that will be presented to you with those decisions, but there are very, very few problems where the data gives you the answer. At the end of the day you’re going to have to use your judgment.

      What “right a lot” refers to is, number one, when those leaders are confronted with those decisions, that more often than not, they pick the right door, but the second part of the definition refers to, more importantly, how those leaders make decisions. Which is, a lot of people think that leadership is about their very strong opinion, arguing their opinion, and winning that argument. And what great leaders do, actually, is they can stake out a position — but they are willing to update (mean change their mind) on what is the right answer, based on new information.

      And you know, even Steve Jobs talked about this at Apple where, some product that they launched, it ended up being a mistake. And it was — one of his reports had been telling him all along, “We shouldn’t do this, we shouldn’t do this, we shouldn’t do this.” And then after it launched and it failed, he came back to that person and said, “Why didn’t you talk me out of this?” And the guy said, “Wait a minute, I tried to talk to you out of it. What are you — what are you talking about?” And he said, “Well, you didn’t do a good enough job, because you didn’t present the evidence to me in a persuasive enough way to make me realize why your point of view was so important.” And so, it’s thinking about it and framing it that way — about bringing forward the right data, and the right information — and then it’s also the job of a leader to solicit that to make the best decision.

      Narratives over PowerPoints

      Sonal: Well that is a perfect segue to a question I’m dying to ask you guys about — because so many companies, their culture is like the mission statement and the values that you’ve shove in a drawer. And so people spend so much time talking about the words and the precision of what they want those principles or values to be, but not actually how to operationalize it.

      So, in that vein, because your book really does outline how to operationalize that through the processes and practices that you guys share. One of the ones that comes up all the time in Silicon Valley folklore is the infamous “no PowerPoints, write a memo.” Let’s tackle this one first, because what you just shared, Bill — about “convince me, share the evidence in a persuasive way” — the point is to be effective and be heard, you have to do that well. So, what is your best advice about how leaders and people in the group can share that information?

      Colin: So, Amazon started experimenting with writing narratives in 2004. And it was a result of weekly meetings, four hours every week with the S-team (Jeff’s direct reports), where 2-3 teams would come in and give either updates on their business — it could be a decision that needed to be made, or investigating new areas to go into. And the business was growing fast, and we realized that we were not making the right types of decisions. Some of the meetings would go over, we never really accomplished what we wanted to.

      I was Jeff’s TA [technical advisor] at the time. We had been looking at other ways to conduct meetings, and we were big fans of Edward Tufte, who’s professor emeritus at Yale, he came to Amazon to speak a couple of times — and after one particularly painful meeting — it was later on in the week, it was toward the end of the day — Jeff said, “Let’s stop doing PowerPoints at these S-team meetings. It’s the wrong tool for what we’re trying to do, and let’s switch over to narratives.”

      Which are really just now a six-page memo. One thing that is a little bit misunderstood is these ideas don’t come out fully formed. They started out as four-page memos — it ended up that six-page was about the right length for an hour meeting. But, we did it because narratives convey about 10 times as much information. You know, the pixel density is about seven to nine times the pixel density. People read faster than people talk, and you can have multi-causal arguments in a narrative much better than a hierarchical PowerPoint. But we realized we just needed a better way to analyze complex situations and make better decisions. So we just experimented with this. And the first ones were not good.

      Sonal: Just to quote you guys, because this actually made me literally laugh out loud. You guys, write, “The first few narratives were laughably poor when evaluated by today’s standards. Some teams ignored the length limit…” blah, blah, blah. And I was like, yes, people are not — I mean, we may be wired to be good storytellers, but writing is actually a hard skill. I do worry that it becomes a bit performative. For the best writers, not just the best presenters, because — one of my colleagues when I used to work at Xerox PARC used to call this “pissing on paper.” Which is, like, this idea of, you know, how dogs piss around their territory. There’s also a real-time component, where people are performing in real time, like, leaving comments in the middle of the meeting.

      Colin: So, a common misconception is, well, now it’s just the best writers instead of the best presenters that’s gonna win out in narratives. We haven’t found that to be the case. It’s really the best thinkers [who] write the best narratives. And some of the best narratives I’ve ever read are by people whose first language is not English.

      Bill: Yeah, in fact, the best narratives, many of them are written by software development engineers who may not have even focused on their writing skills. A good narrative, it’s a very data-based and fact-based document. And, writing good narratives is way harder than making a good PowerPoint. And I think, honestly, a lot of companies don’t do this just because it’s hard. When Colin and I brought this to other companies, what tends to happen mostly is that the author will vomit out about 25 pages of just sort of raw data, at you. Getting that person to then shape it and narrow it down to six pages of well thought out narrative is really hard. And oh, by the way, if you spend 10 hours a day reading detailed narratives, I’ve got to tell you, it can be mentally exhausting.

      But when people muse, like, how does Amazon do it? Like, how is it possible that they can effectively manage such diverse businesses? One of my number one arguments is that they use narratives to conduct meetings, not PowerPoint. And actually, a former colleague of mine (Derek Anderson, who is now the Chief Financial Officer at Snap), I think he made the observation once that Amazon has like a “narrative information multiplier.” It’s a strategic advantage that the company has over other companies, because <Sonal: love this> their executives are, like, 7-8 times better informed about what’s happening in their company, and they’re able to give super granular, specific feedback to those teams.

      Colin: It allowed the S-team to stay connected at a much deeper level. As Amazon started to move into more and more businesses, the span and control of the S-team didn’t grow by 100X — it’s still a relatively small team, and they are just as involved when it was a small company, as they are now when it’s a large company.

      The other thing I’d add about narratives is it does remove a lot of bias from the process, where you can have a charismatic speaker with a so-so or even a bad idea that convinces your company to do something that you should not do. You know, the converse is also true. If you have a shy engineer with a great idea, but it’s a boring presentation. It’s one way in which Amazon removes bias to make better decisions and the idea floats to the top rather than how good of a talker the person is.

      Bill: And we could go on about this forever but, the other part is it actually is a great way to get your team more engaged from top to bottom, especially in a COVID time. The way that these meetings work is you share the document — and you know, whether it’s with G-Suite or with Word — then everyone can then use the comment feature, and it doesn’t matter whether the person making the comment is a C-level person or a fresh-out-of-college individual contributor, all those comments get seen and heard.

      And then when you get into the discussion phase, all those people actually can then understand, you know, why we’re making the decisions we’re making. When you do a PowerPoint, you have to wait to get to the punchline, and while you’re waiting, you’re not sure, like, well, what are we actually doing in this meeting? With a narrative, you just take all that in, and so then you can have high-quality discussion versus that interruption and disjointed conversation you have with a PowerPoint. And if you missed the meeting, you can just read that document. So, there are so many ways in which the narrative method is superior to PowerPoint; that as you can tell, we can never go back.

      How narrative meetings work

      Sonal: Okay, so how does one do meetings then, based on these memos? One of the things you guys said is that sometimes it’s shocking, because the first 20 minutes of a meeting would be silent. And that made me chuckle, by the way, because one of the cofounders of Roam, the note-taking app, was sharing that sometimes they have entire meetings that are silent, because they’re just sharing notes with each other in Roam.

      Colin: Yeah, so Amazon meetings with narratives, they are strange to the uninitiated. It’ll be chit-chat before meeting when they were in-person or it’s online, people are starting to come in — but then there’s silence for about 20 minutes. And during those 20 minutes, people are just focused on reading. There’s no sound. They’re entering questions, and sometimes the presenting team can answer the quick questions right in the comments.

      So, for that 20 minutes, really there’s a huge transfer of information that you can’t see, and then the rest of the 40 minutes is really just high-quality Q&A and discussion on the questions that have already been entered or that come up. The six-page narrative is a forcing function to where you can cover that amount of information in a one-hour meeting.

      Sonal: Tell me a little bit more about what needed to change in the specifics of meetings, in terms of running them. Because the premise of this conversation is — not everyone starts out as Amazon, and that startups can do these things — I’m trying to really tease apart, like, are we just replicating the meeting dynamic in the memo and then the same thing happens anyway? People would begin reading in the meeting, but then how would the decisions happen? Like, what happens next?

      Bill: There are a variety of different kinds of documents you’d review in a meeting. It could be an annual operating plan, it could be a monthly business review, it could be a PR FAQ about a new product. So, the conversation, literally what you do once people stop reading is you just go page-by-page. Or alternatively, if it’s a smaller group, and let’s say there’s just, you know, 10 or 15, you could literally just go around the room and say, “Okay, Sonal you’re first. What questions and comments do you have on the document?” And so, you would get the feedback, questions, and comments from all participants. And then the document itself would present some sort of specific proposal. It’s asking for some budget amount. It’s asking for, are we gonna launch this new product? And, at the end of the meeting, you are tying it up and wrapping it up and saying either, “No, we agree that, what you’ve written in this document, that plan works. Approved, go ahead,” or you debate and discuss it.

      One of the things you might do at the end of the document is make a clear section after you’ve presented all the facts to say, you know, what are the decisions we need to make today? And then list those out. Or, what are the important parts of this plan where we need your feedback and input. Like, we’re not sure whether we should go down path A or path B. A good document will clarify what are the decisions we’re gonna make.

      Colin: The one thing I would add is that a lot of first-timers to narratives, right after people read, they say “Let me walk you through the narrative” and you stop that right away. You know, you’ve just had that 20 minutes of high-fidelity bandwidth — why dumb it down with a two-minute verbal walkthrough of the document? You wanna get to that feedback loop as quickly as possible, just talking about the document.

      Sonal: We didn’t actually really say what goes in the memo. You observed that the memos can vary in form and format by function, and purpose. But can you at least describe what goes in the memo specifically? Like, even the ingredients would help.

      Colin: Amazon has different types of memos for different purposes. So, if it’s some monthly, or quarterly, or annual review, there’s the typical “what were the key wins, what did we do wrong, and what can we do better” and “what are the key initiatives coming up for the next year.” You can have appendices, too, and the appendices are supplemental data that everyone’s not required to read in the narrative meeting, but if you do need to go jump to it to answer a question, you can.

      Bill: It would include tables, with, like, here’s a summary of our financial results from the prior year. Here’s the table — the summary of our plan for the next year.

      Colin: They also work very well with design, which you may not think about at first. But if you’re going over mockups — either UI mockups for an app, or physical prototypes of some hardware, or a process that you’re gonna build — having a narrative actually helps set the stage to say before we take a look at anything, here’s what we’re trying to accomplish with the user experience. Here are our goals, here are the challenges that we’re trying to solve. You know, I’ve been at mock-up meetings where everyone thinks they’re a UI expert, but if you don’t have that <Sonal: Yeah, oh god> before and the comments on, you know, three or four things where you should move this over here. But you don’t do that if everyone’s on the same page with reading a short narrative beforehand.

      Another thing that can be in a narrative that’s really powerful — especially for something that’s going on on an iterative basis, as if you’re refining an idea over time — are tenets. And those are really — okay, what are the design criteria that we are not gonna compromise on, or that we’re gonna fall back on when we have to make tough decisions. And that’s front and center, usually up in the beginning of the document. “Before we go over Amazon’s pricing policy, I want to remind you, here are the four tenets that we’re following to make the following decisions.”

      And you know, to get to those tenets, it was difficult. It took several meetings just to say these are the correct set. That’s a good caching mechanism, because if you’re a manager at a company that uses narratives, you’re gonna be context-switching and reviewing multiple ones every day. And sometimes you may only meet with the team once a quarter, and you wanna be able to very quickly get up to speed.

      Bill: There’s another important technique where you can actually add at the end of the document an FAQ section, for frequently asked questions. So, you’re actually anticipating the kinds of questions the audience are gonna ask you. And a talented senior leadership team is gonna ask you hard, probing questions about things like, “Well, you say in your plan that you’re gonna go do X, but it seems to me that there’s an important hurdle here of — you’re gonna need this important partnership, <Yup> you’re gonna — you have this important dependency. What gives you confidence that you’re gonna — actually can solve that problem?” And you would answer it. Not only does that help reflect whether the author and the team have good mastery of the issues, but it also helps speed up the discussion and the decision-making.

      A lot of my work leading — you know, Prime video was making very expensive multi-year agreements with motion-picture studios to acquire their films and TV shows for the service. These were big numbers, lot of money involved. If we’re gonna go spend that much money, we’re gonna go write up a deal memo that describes, like, okay, we think we should acquire these films and TV shows. Here’s what the package is worth, here are the detailed metrics that show why we think this is a good investment — or why not, in some cases, because sometimes we would review it and say we’ve looked at this deal and we’re not gonna do it and why. And again, just because it’s a Word document, doesn’t mean you can’t put in a chart, a graph, some independent input, an Excel table — all those things can still be in there, but then the narrative needs to clearly state with a clear beginning, middle, and end, like, here’s what we’re proposing to do, we’re not proposing to do, and why.

      Sonal: Yeah, I think that the thing I find most compelling is how much this mimics how writers think, obviously. One of the things that I found when I first came to a non-media company — and was working at one previously — was oh, my God, I have to make so many freaking PowerPoints, and I hate it, because it’s a muscle that’s not ideal for storytelling as much as people think it is. It’s not, it really isn’t.

      But then next, what I love about what you’re saying there is that the FAQs is actually the equivalent of doing the “inoculation technique” — which is what really good op-ed editors will really build in — and that’s not just because you’re trying to inoculate the counterparty to the arguments they’re gonna make — it actually, what you’re really saying there, and I wanna pull this thread, is — you now can have a better, deeper discussion, because you’ve laid to rest all the common things that you can literally get out of the way in, like, a memo in 20 minutes.

      Bill: Yeah, I mean, in a good document, that all comes out.

      Amazon’s guiding tenets

      Sonal: I wanna actually go back to the tenets for a quick second, and then pick up on a thread that you brought up. One of the things, Colin — you described it as a caching mechanism for the leaders, where they — you have to do a lot of context-switching so they get to kind of revisit that cache when they have to get back into that context.

      Colin: Yes.

      But I was thinking about it from the point of view of the group in the room, not just the leader. And how it’s — really sets the shared context for the room, to have the most collaborative mindset possible, while disagreeing within that framework. And in your book, you write that “tenets give the reader an anchor point from which to evaluate the rest” — but here’s the best part. “If the tenet itself is in dispute, it’s easier to address that directly rather than take on all the logical steps that derive from that position.”

      And sometimes you guys spent meeting after meeting just debating to get the tenets right — which I think is really great, because it allows you to actually separate the thing that the tenet is, and then what flows from that.

      Bill: Yes, I spent 15 years at Amazon, and I’ve since gone on — and one of the things that shocked me was that certain topics get, like, recycled for debate constantly, right? <Sonal: Yes. Yes, exactly> And I was like, whoa, we don’t do that. We didn’t do that back at Amazon. Well, why is that? And one is the use of a tenet, which is that all that debate and discussion can land on a fundamental issue about, like, what this company should or should not do. And if you don’t come to a common agreement on it, then you will be constantly — you’ll waste so much time with relitigating these issues.

      The second reason is that what the narrative process forces you to do is by actually putting down on a piece of paper, not only the tenet, but then, like, so here’s the specific plan — then, you set the date for the meeting and everyone reads it, and then you decide. And, I realized that a lot of those things get relitigated. So, it was one of the ways [in] which Amazon was very effective, which people don’t realize, as a management company.

      Sonal: Yep. On that note, does it then serve an archiving function for new hires and onboarding, that you scale that tacit knowledge that’s been made explicit? How does that work?

      Bill: Yeah, great point, because the other thing I also saw was whenever you hired someone new in — within a matter of a week, they would want to go relitigate. They too, <Sonal: Yeah, totally, it’s exhausting> would trip across, well why don’t we do blank? Or why don’t we — and it’s like, oh my gosh — and you can just say, “Look, here’s the tenet, here’s the document. You know, take a look at this, and then come back and, you know, talk to me.”

      Starting with the press release

      Sonal: Right. And then when you do decide to relitigate, it’s an actual intent <right> versus an accidental, every-new-hire-repeating-recycling <Bill: Right> the conversation. Okay. So the other big Silicon Valley folklore when people talk about best practices from Amazon is this idea of working backwards from the press release. Now, I know that you guys talk about “working backwards” well beyond — it’s the reason it’s the title of your book, obviously — but I really want to probe specifically into the mechanisms of what that means. Because, like, how does that happen? Is it just, “Oh, I wrote a press release for this thing I wanna do” — it’s a product, it’s an idea, it’s a service. I really would love to hear what, where, how, and why. And also, I’d love to hear the origin stories, if you have any specifics there as well.

      Colin: At its heart, the working backwards process is really starting from the customer perspective — and everything you do works backwards from that. The PR/FAQ (the Press Release and FAQ), that is the tool that Amazon uses to achieve the perspective of starting from the customer. It’s different than how a lot of companies develop ideas and products. A lot of companies use a “skills forward” approach. What are we good at? What are our core competencies? What are our competitors doing? How do we nudge into this adjacent market? How much market share can we get?

      Bill: When I was in business school, and taught to think about, like, how you expand and grow, as Colin already described — you create a SWOT analysis.

      Sonal: What is it, the strengths, weaknesses, opportunities, threats, right? Right.

      Bill: Right. But there’s no “C”, there was no customer.

      Sonal: Right.

      Colin: Amazon has put in deliberate mechanisms to make sure that the customer is front and center from the very first iteration of an idea. So, if anyone says — raises their hand and says, “I have a great idea,” the first thing that the manager or the person in the group will say, “That’s great. Why don’t you go write a working-backwards document.” And what that is, it’s two things: it’s a press release, and then a frequently asked questions document. And the press release has to clearly explain to the customer what it is you’re building — what’s the problem you’re trying to solve for the customer, and why does it make their life better? It can be something very small, or you know, it could be moving into a brand new industry. It’s a fractal process, which is great.

      Another thing is, these PR FAQ documents — it’s an iterative process. First of all, most of the ideas that go through it don’t make it out on the other side. And second is that it takes several, several iterations and feedback to refine before the project gets green-lit.

      Bill: In fact, the origin story of, like, how we got to the PR FAQ — or at least a part of it — both Colin and I were present for this, because in 2004, I landed on a new role working as one of the founding members of the digital media team at Amazon. For perspective, at the end of 2003, 77% of all of Amazon’s worldwide revenue was media products. But it was all physical media products. It was books, CDs, DVDs, VHS tapes. And, the writing was on the wall that, like, this business is not here to last. That — there were already a couple of million iPods that were sold. Millions of people were using Napster to file-share.

      Sonal: Napster, right.

      Bill: Right? It’s pretty clear that, like, okay — now that we have the internet, it’s just a matter of time before people, you know, consume their media digitally. But we didn’t know how. So, what I did — pulling out my bag of tricks from business school — is [I] marched into meetings with Jeff where, like, here’s our projections for how big the e-book business will be over the next 10 years. And here’s our projected market share. And, here’s what the pro forma P&L looks like. And here’s the kind of deals we’ll make with publishers. And here’s the competitive landscape.

      And, I was so proud of all this work, and he looks up at me and says, “Bill, where are the mock-ups?” And I didn’t have any mock-ups at that point. I was doing, you know, like, “Oh, here’s the projection — I’ll get started and we’ll start working on launching, you know, a better e-book store.” And, by “Where are the mockups?” what he meant was, this is all super interesting — or not really actually, is what he was saying — but what’s more interesting is like what is gonna be the customer experience?

      And more to the point, as we started to debate and discuss different ideas — whether it be in digital music or e-books — the discussion was about, well, why would we bother building, like, a me-too versus what you know Apple’s already got the iPod and iTunes. So what’s in it for the customer to have just have another — a knockoff of that service? Like, what can we build that actually creates real value for customers, something new we should invent on their behalf?

      And we tried mockups for a little while, but frankly there were, like, so many questions and so many details that we hadn’t thought out, and Jeff one day said, “Okay, I got a better idea. Why don’t we — everyone in this room — write up the idea for what they think we should go build in digital? We’ll write those things up, and we’ll come back in a few days, and we’ll read those, and we’ll go from there.” And this is before we’d done narratives. We had not, you know, figured out this PR FAQ concept, yet. But once we did that, everything changed. And suddenly we were reading, you know, one document was describing, like, a puck that would sit on your countertop, and you can talk to the puck, and could order groceries from the puck. I described the document about, you know, what we might go do in digital music. Another one was describing an early version of what Kindle might become.

      But some of these documents were, like, eight pages long. We needed to get this to be, you know, more pithy and clear. And Jeff said, “I know. Let’s write the press release for each one of these ideas instead.” <Sonal: Ooh, neat> And he said, you know, we should read the press release, because normally that comes last. And he said, you know, normally the engineering group and the product group, they go and they come up with the idea for the product, and then at the very end, when it’s time to sell it, they chuck it over the wall to the marketing team and say, “Okay, figure out how you go sell this thing.”

      And he said, but what if by the time that thing got to the marketing team, they said, “Yeah, well, the thing you built — for that really to work, we actually need that to have a price point of $99. But you’ve built it in such a way that it’s costing us $150 to manufacture each one — so, yeah, we’re not gonna be able to sell too many.” In other words, if you had known upfront that you needed to hit a BOM [bill of materials] of less than $99 to make this product go, then you would have designed the whole thing very differently and understood the constraints. It might have a whole lot of — there might be vaporware concepts. There might be concepts that you don’t know how you’re gonna solve. Maybe business model problems — but then in the FAQ, then that defines, okay, what are the hard problems we’re gonna have to go tackle to make this exciting product a reality?

      So, we switched to that method, and it was halting progress, and had we not really taken that approach, we would have not ended up with what turned out to be a breakthrough product, which was the Kindle.

      Sonal: That’s fantastic. So, a couple of questions to just quickly probe on a few nuances: First of all, I know what you’re really saying is, flip the perspective from inward-out to outward-in — which I think is really interesting, because that’s something I constantly think about content. Like, orient it in the value to the reader. But what would you say on the flip side, to the crowd that often says that part of the problem with “working backwards” [is], “Well, then you’re not really inventing what they don’t know what they want.” And how does one write a press release for the startups out there thinking, “Oh, well, you know, we’re creating a new category — this is not something that exists.” Tell me more about how you might address that crowd with this PR FAQ approach.

      Bill: I would submit that, in fact, that’s what this process is actually designed for. In 2004, when we started on digital media, the way e-books were, you could only read them on your PC. There were no, like, offline readers and tablets and devices in those days. They were priced way too high — like, the same price as the hardcover book. The e-books had existed for four or five years before the Kindle launched in 2007. But it was a tiny, tiny business. And it was a tiny business, because no one had imagined, well, what do I need to create, what’s the new thing I need to build to make e-books work?

      We defined what would be the ideal reading device — everything from, you would be always connected to the internet (which by the way, devices weren’t that way back in 2007). That you wouldn’t have to create some separate account or link your account to some mobile carrier. That when you got it out of the box, it already knew who you were, and so if you had actually bought a bunch of e-books online, they were magically already uploaded onto your device. All those kinds of things would be described in the FAQ section to say, “Yes, here’s hard problem one, and here’s how we’re gonna solve this problem” — or, “we don’t know how we’re gonna solve this problem yet, but here’s our path for how we’re gonna go work on that.”

      Colin: Yeah, I would just say, there are two very real use cases where Amazon used the working backwards-process to create something from scratch. With AWS, you know, cloud computing didn’t exist. And as we’re working through the Kindle issues — you talk about context-switching — an hour later we’d go to another conference room, and we’d be with Andy Jassy (who’s the CEO of Web Services), and we’d be reviewing Word documents about what would eventually become cloud computing.

      And it took us about two years to come up with, what are we actually trying to create? And you know, the first two were centered around storage and compute, eventually. And the press release — one area where it really helped to crystallize everyone’s thinking is — we came up with the saying that we want people in a college dorm room to have the same access as an Amazon developer to world-class infrastructure. And that <Sonal: Wow…> was just a powerful metaphor about, okay, so what are we creating — and, you know, what is this infrastructure that we really need?

      And starting from the customer experience, we had many, many small teams throughout the company just screaming at Jeff, at the infrastructure team, “I built my service, it’s taken me too long to deploy it and get it out and ready for customers.” And that’s where it started off as provisioning, but it kind of morphed through this working-backwards process to compute. And then in terms of storage — there’s a whole bunch of different types of storage. It narrowed down to simple storage service, was the very first thing, and then we would build out from that.

      Sonal: I love that example, because it really emphasizes this point, that when you start with the PR FAQ, you’re essentially starting with the differentiation. Because you’re really thinking in terms of the value — because there’s a million options for people to pick from — so that’s when you go from provision to compute.

      Because you’re thinking, if the customer is this kid in the dorm room having access to — and we often talk about the power. Like, actually Marc Andreessen, in his original 2011 “Why Software is Eating the World” op-ed, points to the power of this movement — like, you know AWS has been huge in bringing — we even have this op-ed about why every company is a fintech company. We actually call it “the AWS moment in fintech” — it’s quite amazing the impact that AWS has had on the industry, and there’s no question about that. But that precise point — of starting with the PR FAQ — to take what could have just been like, “Oh, here’s some storage, and here’s how to provision the services you need,” to here is how to create an entirely new business, that’s a whole different game.

      Lean startup concepts

      Bill: The other thing that’s really important to note, where the PR FAQ process is misunderstood, or, in conflict with the startup community today, is the lean and agile approach.

      Sonal: I wanna hear about this. Love it. Cause some fights, Bill.

      Bill: Here, let me tell you what the problem is with the lean and agile approach.

      Sonal: I love Eric Ries, for the record. He himself is the first — and he’s actually said it on this podcast, that sometimes people get a little cult-y and follow the letters of the rule instead of the principles of it. I actually personally am a big believer, having worked at Xerox PARC, in the maximum viable product sometimes instead of the minimum viable product.

      Bill: Yeah, it’s really — actually it’s the v part. The problem is that people focus on the m, the minimum, and they don’t focus on the viable. So, what is the definition of viable?

      Sonal: Yeah, what is the definition? What would you say it is?

      Bill: To me, the definition is like, “Oh, if I go build this thing, I have created a — insert size-of-business here — $100 million, billion dollar business — I have enabled, if I’m right, then this opens the path for, like, something really big. And I see instead happening, “Okay, I’ve got a sprint, I’ve got a couple of weeks. What can I get done in a couple of weeks?” Or, “Oh, what can I launch quickly to sort of test and learn?” Right, where then, you’ve created completely different constraints. And oh, by the way, if your whole dev team thinks with their whole roadmap, and breaks it into these little chunks of how do I you know iterate quickly, test, and learn, then you’re gonna launch a lot of small things where the actual size of the viable business on the other side of it might be sized in the one-million dollar range or two-million dollar range, or, like, the actual potential good outcome is very, very small.

      Now, there are plenty of places where this approach is totally applicable. Search, where you’re, like, how do I test and learn with, like, changes to the algorithm? Or changes to the logic, or a new AI model. But if you’re thinking about, I’m starting from scratch, I’m trying to create a new business — the problem with this MVP approach, where the viable part isn’t really thought out, and mapped out, is that people haven’t really thought through, like, well what could this really become, and why might this not work? And in many cases, they could have actually — if they instead spent more time upfront in the planning process — thought much bigger about what does the customer really need — or wow, I’ve really created some significant value versus frankly, these little incremental changes that don’t really even move the needle at all.

      Sonal: People don’t do press releases for incremental releases, they do press releases for big advancements.

      Bill: That’s actually an excellent point. If you parachute into a new company, and you look at their product roadmap, and there’s not one thing on there you’d write a press release about — then, like, you’ve got a problem.

      Sonal: Yeah.

      Colin: To me, viable means you read the press release of what you’re trying to build, and you want to buy or use the product. If it’s not something that you wanna buy or use, it’s not viable.

      Sonal: Yeah. I mean, I would also push back on the definition of viable, because what I heard from Bill — and Bill, you should correct me if I’m wrong on this — Colin, when you say like that obviously the customer is gonna wanna use it, but, to me viable is not just that it’s something a customer would use <Bill: Right!> because I think there’s a lot of dumb things customers would use, quite frankly. I heard it more as the enabling conditions to really make something bigger, and then also to address a deeper, more underlying opportunity sometimes instead of the surface opportunity. Because — not to get all jingo-ish on this — but I, of course, think of Clayton Christensen’s Jobs-to-Be-Done framework — and it’s sort of about, like, what is the underlying job to be done that is being fulfilled for this person? So, that’s how I heard the viable — what does it take to make this happen — whether minimum or maximum — and then also not only thinking about the product, but all the enabling conditions to get to addressing the deeper opportunity.

      Bill: Yeah.

      Colin: Another way where I’ve seen that MVP process being misused — because it can be useful — is that the process itself becomes the goal. And because I’m supposed to launch every two weeks, or I have an MVP so this MVP card is “go to the front of the line and get whatever resources I want because I’m doing an MVP.” So, don’t let the process itself become the end goal, and then don’t let the MVP get in the way of understanding your customers. Because you know, some people actually forget the customer problems that they’re trying to solve.

      Sonal: Which is the whole point of doing an MVP in the first place. I mean, the whole point is to be able to get that feedback so quickly. Because the whole reason that that idea came about was that, you don’t wanna have that very long delay in the feedback loop.

      Colin: Yes. And if you’ve watered down the feature set so much just to conform with what the process is, it’s a shallow, pathetic version of what you’re actually trying to test. And so you say, nothing here — when you never really gave it your best shot.

      Sonal: What’s really interesting is you actually, in your book, outline the total addressable market, or TAM. And I found that to be very interesting — especially in connection to the discussion about how to think about the biggest possible market for this product, working backwards.

      Bill: Yeah, we spilled a fair amount of ink on the TAM part in the book, because people with less experience sort of get this part wrong — to not only think about how many people have this problem, but like how big is the need. And for how many consumers is this problem really big enough that they’re willing to spend money to do something about it? How much would they be willing to spend? And, you know, how many of these consumers actually have the characteristics, or capabilities to actually make use of this product — and, a lot of people don’t really step through that clearly.

      Colin: The thing is that you wanna ask all of the hard questions upfront. And as we talked about, this is an iterative process. So, in the review meetings, you know if there are a couple of questions that you don’t know the answer to — you just append those to the document and say great, let’s come back in two weeks — however long it’s gonna take. And the FAQ, you can basically break it up into two different components. One is an external FAQ — that you’re explaining how does this product work, how much does it cost, why should I use this service versus what’s out there on the market, or why do I need to change my behavior. And then the internal FAQs — what are the things that we need to go organize and solve in order to make this product or feature a reality.

      And then, like you touched on — how big is this opportunity? You know, what is the total addressable market? Then you can size up the opportunity, and that’ll tell you whether it’s worth doing or how much to invest in it. Because one of the things that you ask when you go through the working-backwards process is, [is] this big enough to be worth doing? So it may be a good idea, but it just may not have the impact and scope on your customers or the organization to make a difference to be worth devoting resources to.

      Bill: When we were figuring out in digital media and AWS what are we gonna go build, and you parachuted into either of those teams and said, like, “Hey guys, how’s it going?” and we said, “Oh my god, we’re so frustrated, because we just wanna go and build this thing and get it out there, but Jeff, you know, is insisting that we go through this PR FAQ process and figure this all out in advance.”

      And I was among the people who found this frustrating. And it was only in hindsight that I realized, like, how smart Jeff was to slow us down. And we co-opted the marine scout sniper saying, which is that “Slow is smooth, and smooth is fast.” Or, “measure twice, cut once” — or sometimes Jeff would refer to himself as “the chief slowdown officer” when teams were sort of itchy to pull the trigger and build and launch something.

      What I came to appreciate in retrospect, is that to build — to actually create value, you really have to take the time and think big — that’s what the PR FAQ process does — and if everyone can go write PR FAQs, and you’ve got 30 or 40 different ones to review, then I’m here to tell you that the best ideas are gonna bubble to the top pretty clearly, based on this process. Whereas the MVP/lean process really is just about cranking stuff out, rather than first filtering, thinking through, refining…

      Sonal: What to do.

      Bill: …what to do. Yeah, people confuse speed and activity, with effectiveness.

      Principles in balance

      Sonal: Yeah, exactly. What I find so fascinating about that, by the way, just coming back to where we started, is how the 14 leadership principles balance each other. Because you describe Jeff in this context, and a lot of these processes as ways to slow things down up front — but then you also have principle #10 (or whatever number it was), for bias for action at the same time. You think of it as a whole system. And it’s really interesting, because it resonates to me with Ben’s — Ben Horowitz’s book on culture — because he describes, like, the value system of the samurai warriors, where they would have something that was like Bushido — where, there’d be something that was incredibly kind and generous, and then something that was incredibly vengeful in the same, like, framework. It’s very interesting how those motions are kind of opposite, but yet in balance.

      Colin: Yeah, they do work together. We started off the conversation about the “are right a lot” principle — well, the one right below that is “learn and be curious.” And the one right above it on the list is “invent and simplify” — and, you need to do both of those in order to be right a lot.

      The one thing I would add — you know, a lot of people say, well, Amazon, they either have so much money or so much time they can afford to do all these things. Long-term thinking doesn’t necessarily mean it takes longer to get to your end goal. You know, Amazon built a $100 billion business faster than any company. AWS got to 10 billion dollars in revenue from zero, faster than Amazon the retail business did.

      So, sometimes, to slow down to move fast — even with smaller organizations, [you] more than likely will get to where you want to be quicker if you take some of these steps. Typically, what you’re doing is you’re doing off-path, distracting activities, that by the way are taking away from your bottleneck resources — which are typically software engineering resources at a lot of organizations <Sonal: Yes, exactly>that are meant to build up long-term value.

      Bill: Yeah, and just to — the AWS thing is so remarkable. Think about that for a minute, because we just told you — I mean, what was it, Colin, 18 months, 24 months? How many months from go, before the team even wrote the first line of code?

      Colin: I mean, it was at least 18 months. And I did have software engineers after some of the meetings come to me and say, “Hey, can you remind Jeff that our job is to write software code, not documents?” and they were, you know, itching to go. It took a while to figure out what we should build, and that time was very well spent writing Word documents versus writing C code.

      Bill: And not only that, it’s empirical fact that it was well spent, because the company set the record for the fastest company to a revenue milestone by spending the first 18-24 months planning.

      Colin: And you talk about distractions, we did not know at that time what other people were doing. You know, web services were no secret. There were other companies who — by their own right — should have gotten there first. We didn’t know if someone would come out with a set of developer APIs at that point in time.

      Sonal: I know. That scares me thinking about that, actually. Like, almost two years planning, like, that scares me.

      Bill: Think how scared they were.

      Colin: But Jeff had the fortitude to say it’s not ready yet. This is not what we want to build.

      Bill: It’s not viable.

      Colin: You know, talk about what makes Amazon special, you can read these principles — sticking to them is sometimes quite challenging, either when things are going really, really well, or when they’re not going well, or when there’s uncertainty. These are not just posters on the wall. They’re woven into the DNA of everyone who’s been in Amazon for any length of period of time.

      Sonal: I’m just struck at all the parallels in all the things you guys are saying — because, essentially, every business today is a creative business. What really strikes me is — I’m not gonna make a comparison between podcasts and AWS, but I will say that I talk to a lot of podcasters [about] how to make their podcast stand out, differentiate. Like, how to do editorial strategy comes up a lot — and one of the things I constantly say is, if you can’t be first or leading, then you have to be very differentiated. It’s really interesting when you talk about the bottleneck resources, I think of things in terms of opportunity cost and return on energy, or what I call ROE. And, similarly, you have to pick which things you’re gonna do for the greatest possible hits, or you’ll never punch above your weight, or be heard above the noise. Which I think is so fascinating;

      We had Jeff Lawson from Twilio on the podcast recently, and he’s, like, you know, we need to give them problems, not just like specs and things to work on — and so when you describe that they’re like, “Uh, we’re spending, you know, 18 months writing planning documents,” that is treating software developers as creatives. 

      Colin: It also creates alignment for the teams then who will go out and build — because sometimes it requires more than one team. If they’re not aligned on what the problem is, they’re gonna solve different problems and those components aren’t gonna fit.

      Advice for startups

      Sonal: Right. I want to switch into talking about some specificities, some advice for startups. 

      Colin: One tool you can use for startups — and I’ve actually done this at other companies — if you’re trying to figure out what to build, you can write competing press releases, with different takes on the problem. <Sonal: Oh, I love that> <Bill: Right> And then, it becomes pretty apparent when you’ve got two or three or four different approaches that, we really need solution A. Or, I’m gonna take the first part of this solution and combine it with a great idea that came up in the second one — and this is actually what we want to build. So, it’s a lightweight way to crystallize your thinking and also get alignment.

      Bill: Yes, think about how inexpensive it is to write a one-page press release versus how expensive it is to build mockups. To Colin’s point, when you read them all, as a group, like, the best ones are gonna be clear.

      Sonal: You know again, this is where I just can’t get over how creative the book is, in terms of its application to all kinds of creative fields, companies big and small — because to me, both the memo strategy, the PR FAQ — and then this idea that you both shared of being able to write competing press release and then harness kind of the wisdom of the group — it’s actually about the power of narrative, for really helping instantiate things that you know when you know. I think a lot about how tools change our thinking and vice versa, and you guys are essentially describing these practices for how that plays out in organizations, especially as they scale.

      On this note, you have a section in your book — the subhead is “Better Coordination Was the Wrong Answer” — because that’s one of the common myths that people have as they scale is, we need to coordinate better. And in fact, it creates layers and layers of crap to deal with. People create entirely new roles, like dedicated chiefs of staff just to, like, you know, create stitching and seaming between teams. And it’s just the most ridiculous thing I’ve ever seen and heard. And I just wanna hear more from you guys, especially because a lot of listeners — and we’re talking about what happens when you scale and grow very quickly — you guys really did see this phase at Amazon.

      Colin: Yeah, this is a case where Amazon faced the same question that a lot of growing organizations do — we’re adding more people, it just seems like it’s taking longer to get things done — but came up with a different answer. Some companies would say, let’s build coordination tools, let’s collaborate better. And Jeff said, I’d love to eliminate coordination altogether.

      Now, in practice, you can’t eliminate it completely — so, you break up into loosely coupled teams. And this was a hard, hard problem to solve, because it required to change the way we built Amazon, technology-wise, to decouple it into what’s now services-based architecture. That was hard to do back then in, you know, 2000-2001, especially as you’re growing super fast. But then there’s also the organizational component too, about how do decisions get made? It was a multi-year effort, to be quite honest, but if you’re gonna go grow 10X and 100X, we’re not gonna spend any time building — we’re gonna spend all of our time coordinating. So, you know, nip it in the bud. And Jeff wanted Amazon to be a place where builders can build.

      Bill: So, the ironic thing is that we started off with the process that was the conventionally accepted traditional wisdom — it was brought in what we called NPI, new project initiatives — it was — it was horrible. I mean, basically it was another example of the process becoming a thing, where you had to come up with, like, what’s your new project initiative? You had to write it all up, you had to project what are the financials behind it — most of which were totally wrong, and guesses, like almost all pro-forma P&Ls are. And then put it in front of a big committee, and try to take it upstairs.

      Not only was this process a massive waste of time, but then you’d have these frustrating business reviews with the team saying, “Yeah we really need to go build X, Y, and Z,” and Jeff and the S team saying, “Yeah, I agree, you know go build X, Y, and Z,” — but they’d say, “Well I can’t, because I need browse to do this. And I need the team that works on the checkout — order pipeline, and they already have these four other projects they’re working on, so I’m just sitting here in line, and I can’t go build it.” And so, you’ve de-empowered your teams. The senior management, they become like referees of, between teams, who’s gonna go do what. And, frankly, it’s not much fun. So, this was a huge breakthrough for the company, was breaking down — you know not only, of course, breaking down the code into APIs — but then breaking down the teams into single-threaded focus teams.

      Colin: There are some roles that don’t actually fit too well in this type of paradigm. So, for instance, like a Chief Product Officer doesn’t really fit in this role, because if you ostensibly are responsible for making every product decision in the company — from what’s going on in the warehouse to what should the — how should the apps be built, to how can we decrease delivery time — there’s no one person who can be obsessed with all of those details. And so, that role typically doesn’t fit as you separate into these small, separable, single-threaded teams. The fallacy is [that] as your company grows, that Chief Products Officer isn’t really doing that anyway, because they can’t get that much high-quality information to make all of those important decisions. You do want the teams closest to the customers making those types of decisions anyway.

      Sonal: Yep. I call that “bare metal decision making”, like, who’s the closest to the metal of the thing. And that’s what I think is super valuable about what you just said, and you’re essentially describing these small units as, like, every team has its own mini – like GM, of every mini unit as a leader, and then every product — you’re, like, so close to the core in that decision-making framework.

      Bill: That’s right, the idea is for each one to be like a self-contained unit. And in search, that’s all engineers — but in some other business like my digital video business, it was like a combination of engineering and marketing people and people working with studios. But you have all of the resources you need — you don’t have these dependencies that basically slow you down.

      Colin: During this timeframe where we’re making the transition, we were using narratives, and one of the questions that we actually required was for the teams to put in “what things that are not under your control that you wish you had under your control” — and how <Sonal: Oh, great!> are you gonna organize and create APIs so that can happen? Again, this is work that’s below the tip of the iceberg of where Amazon really put a lot of effort into. How can you still grow and be as nimble and as agile as we were when we started out. And, it’s not easy — but, you know, it only gets harder, so you may as well start now.

      Bill: Yeah, and don’t take away from this that Amazon was some Nirvana world, where you never had to worry about coordination with other teams. It was just that we worked so hard to minimize the degree to which we did.

      Sonal: Yeah. I’m very fascinated by how organizations evolve in general, to be effective, and when you think of any large company, it becomes a complex system. And you’re essentially describing how modular, self-contained units can thrive in these complex systems. It’s a lot like evolution, really.

      Bill: And these pieces all fit together, because then we have to move back to like, oh, the reason that Amazon could do that is because those teams wrote narratives and PR FAQs, so they made it abundantly clear. If Jeff wanted to see what is this team doing, “Oh, here you go. Here, read this document” — and in, you know, two hours, he knew exactly what they were doing.

      Sonal: Right, he makes him the chief ecosystem officer, essentially, not just the chief slowdown officer.

      Shadowing Jeff Bezos

      Okay. So, Colin — so a question I have for you. It’s really unique that you were able to shadow Jeff Bezos, and be his shadow. And you know, I don’t wanna make this about the glorification of Jeff Bezos. I’m so not interested in that, because I think there’s just plenty of narratives out there — what I am interested in, however, is this question of what it takes for a CEO and a leader like that to evolve. And what you saw in shadowing him on that front, and then also the act of shadowing. And that itself as a mechanism for learning, mentorship, apprenticeship, if you will. I’m very fascinated by this whole thing.

      Colin: Sure. So, I mean, my role as Jeff’s shadow was primarily two parts. One was just to make him a more effective CEO on a daily basis. Making sure that the right issues are surfaced, that the people coming into the meetings would cover the right topics and have enough information to make the right decisions. And afterwards that it gets followed up on. So, you know, kind of bookending the day. But then the second and more important part, the longer-term part, was it was a training role. And the way Jeff put it is, I want us to be able to model each other, and you know how we would think in different situations.

      One of the enlightening things that I realized during my time with Jeff is what he chose to work on during the two years I was his shadow or technical advisor. And it wasn’t the biggest businesses at Amazon — about half of the time were spent on what would become AWS and Digital, and those businesses had revenue of effectively zero for, you know, those two years. And he told me more than once, you want your top leaders and your most impactful people working on your biggest opportunities. And that may be different from what your biggest current set of activities are.

      How Jeff changed during that time and just to become a better CEO — one thing he said is, he learned how to become a better operator, becoming more operationally efficient. Fortunately, that is a teachable and learnable skill. And Jeff had some great people at Amazon, like Jeff Wilke, you know, who’s a great operator. He’s the CEO of the Consumer Business. And now in his own right, he’s this great operator and insists on the highest standards — it’s one of Amazon’s leadership principles. He holds people accountable, himself included, by the way.

      And I would say that I also learned that sticking to these core principles is harder than it sounds, but the time when you need them most is the times when it’s easiest to ignore them. So, for instance, Amazon Prime, he made the call that, “Hey, we’re gonna go launch this shipping project in the holiday season.” And it was not very popular at the time, but when he explained his thinking, that you know customers were basically giving us a B minus on our 3-5 day shipping — even though we had just gotten reasonably competent and spent a couple of a hundred million dollars doing that — Jeff looked at it, you know, long term and said, “Well, we’re becoming a smaller and smaller share of the overall ecommerce industry, so we’ve got to make the change now.”

      So, it wasn’t that Jeff had this insight that comes once in a generation for Amazon Prime. He just stuck with the leadership principles and took them to their logical conclusion. When it was tough to do in the face of the holiday season, of what our quarterly results were.

      Sonal: By the way for the listeners, because we don’t obviously have time to go into the whole book, but, you know, half of the book — the first half is about the principles and these practices and mechanisms — but the other half is actually showing, through these very detailed case studies and examples that you both participated in or witnessed or oversaw firsthand, in this invention machine at work. And that includes the Kindle story, the AWS story, Prime Video, and what you just mentioned, Colin, which is Prime. And what’s really interesting in that Prime section — is that not only that Jeff had the wherewithal to push through — but what’s really interesting in the Prime program story is that it’s all the iterations involved to actually get it to what it is today. Because there was, like, a 1.0 and then, you know, the loyalty program it later became. So, I think that’s a really great part of the book, for those that want to learn more, and — Bill, anything to add there on what you saw on the evolution?

      Bill: I do, in fact. Jeff — I remember Jeff, at least more than once, talking about the way a leader needs to evolve as the company grows. And I think about this frequently, which is — he said, at the beginning, the leader needs to really focus on what. Then they really need to focus on how. And then, eventually their focus really just becomes who. <Sonal: Wow.> What is, you know, what is our business? What’s our product, what are we building, what are the details of that? How is, what processes? How do we do the work? What is the filter, the lens through which we make decisions? And then who, of course, is who are my leaders? Who, how have I — making sure that I assembled the right team — and in doing so, how do I delegate responsibility to those people so that they can carry out those details?

      That is a classic and challenging transition for any entrepreneur/owner, who starts off being in control of every detail, and that they have to slowly let go of those details. And then they have to figure out how they put in place the right mechanisms to be able to properly audit the work of the teams. And that’s what, you know, we tried to describe in the book, is actually this management science, frankly, that Jeff and the team developed to really solve this problem.

      Sonal: “Working Backwards: Inside Stories and Secrets from Inside Amazon” by Colin Bryar and Bill Carr. Thank you so much, you guys, for joining the a16z Podcast.

      Colin: Thank you.

      Bill: Thanks, Sonal.

      • Colin Bryar

      • Bill Carr

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Anatomy of a Hack: SolarWinds and Ripples Beyond

      Steven Adair, Joel de la Garza, and Sonal Chokshi

      In this special “3x”-long episode of our (otherwise shortform) news analysis show 16 Minutes — past such 2-3X explainer episodes have covered section 230, Tiktok, GPT-3, the opioid crisis, more — we cover the SolarWinds hack, one of the largest (if not the largest!) publicly known hacks of all time… and the ripple effects are only now starting to be revealed. Just this week, the U.S. Cybersecurity and Infrastructure Security Agency shared (as reported in the Wall Street Journal) that approximately 30% of both private-sector and government victims linked to the hack had no direct connection to SolarWinds. So who was compromised, do they even know, can they even know?!

      Because this hack is a supply-chain compromise involving various third-party software and services all connected together in a “chain of chains”, the knock-on effects of it will be revealed (or not!) for years to come. So what do companies — whether large enterprise, mid-sized startup, or small business — do? What actually happened, and when does the timeline really begin? While first publicly revealed in December 2020 — we first covered the news in episode #49 here when it first broke, and there have been countless headlines since (about early known government agency victims, company investigations, other tool investigations, debates over who and how and so on) — the hack actually began not just a few months but years earlier, involving early tests, legit domains, and a very long game.

      We help cut through the headline fatigue of it all, tease apart what’s hype/ what’s real, and do an “anatomy of a hack” step-by-step teardown — the who, what, where, when, how; from the chess moves to technical details — in an in-depth yet accessible way with Sonal Chokshi in conversation with a16z expert and former CSO Joel de la Garza and outside expert Steven Adair, founder and president of Volexity. The information security firm (which specializes in incident response, digital forensics/ memory analysis, network monitoring, and more) not only posted guidance for responding to such attacks, but also an analysis based on working three separate incidents involving the SolarWinds hackers. But how did they know it was the same group? And why was it not quite the perfect crime?

      image: Heliophysics Systems Observatory spacecraft characterize, in the highest cadence, the constant stream of particles exploding from the sun affect Earth, the planets, and beyond via NASA Goddard Space Flight Center / Flickr

      Show Notes

      • An overview of how SolarWinds was hacked [2:21], the attackers’ methods [5:33], and their impressive sophistication [8:17]
      • A step-by-step explanation of how the attack took place [14:20] and how the hackers avoided detection [21:18]
      • Open discussion of what the experts know so far [23:52], including how we know the attack was coordinated by the same group [29:21]
      • Big picture security questions [33:26] and how businesses and consumers can protect themselves [42:51]


      Sonal: Hi, everyone. Welcome to this week’s episode of 16 Minutes, our short form show where we talk about the news, tech trends in the headlines, tease apart what’s hype/what’s real, and where we are on the long arc of innovation. I’m Sonal, and today’s episode is actually one of our special “2-3X” long explainer episodes — which I’ve done every so often for topics that keep coming up over and over in the news (most recently on Section 230 and content moderation, previously on TikTok, and even earlier on on the opioid crisis). You can catch all those at a16z.com/16Minutes. But today, we’re covering the SolarWinds hack, one of the largest (at least publicly known) hacks of all time.

      Not only has it been in the news a lot since it was first publicly reported in December, with countless headlines since — but the most recent report, from the acting director of the Cybersecurity and Infrastructure Security Agency, was that approximately 30% of both private-sector and government victims linked to the hack had no direct connection to SolarWinds, as reported in the WSJ just yesterday. So, it’s gonna have ripple effects for quite some time.

      So, we’re doing an “anatomy of a hack”: a teardown of the specifics we know so far, what went down, and what we need to know — whether big company, small company, or individual.

      For quick context before I introduce our experts: Over 18,000 customers downloaded compromised software, though it goes well beyond them. Those customers include several large government agencies (which we covered last year on this show). Private sector victims include companies like Cisco, Intel, Microsoft, NVIDIA, Deloitte, VMware, Belkin, and others. The broad consensus – per a statement issued by the Office of the Director of National Intelligence, the FBI, Department of Homeland Security, and National Security Agency – is that Russia was most likely the origin of the hacking, and more specifically, that the Cozy Bear group (also known as APT29, overseen by Russia’s intelligence service) was responsible.

      That’s just a super high level, because we’re actually gonna go deeper to break down the who what when how – and the chess game of it all. So now, let me quickly introduce our experts. Our in-house expert is a16z operating partner for security and former CSO, Joel de la Garza. And our special expert guest is Steven Adair, the president of Volexity, an information security firm that does incident response and forensics (including memory forensics), and they’ve responded to multiple cases of this. Their team actually put out several detailed posts on it and more.

      Overview of the SolarWinds attack

      Sonal: But first, Steven, can you summarize what happened? Obviously we’ll continue to dig in on the details throughout the episode, but the reason I’m asking is, I’ve started to lose track. I bet a lot of our listeners are getting a little inundated with this headline fatigue too, like — now this, now-what. So, tell me basically what actually happened. What do we know?

      Steven: Yeah, sure. So, SolarWinds is a company that creates network and system management software that’s used really heavily by tens of thousands of organizations around the world, so it’s used by large giant commercial companies, Fortune 500. It’s used by small organizations, managed-service providers, and governments. So, it’s a piece of software used to manage these, like, really sensitive important assets. So, think about the IT teams, and people who wanna watch what’s going on key systems, on network devices, and things that are really important within a network. They have a product called Orion, that’s their flagship product.

      And what happened is that SolarWinds was basically breached. How exactly, you know that’s not really been published. We don’t know. But attackers were able to compromise SolarWinds, get into what’s called, like, the “build process” of this product. So essentially, the development or the software that’s downloaded and used by all these organizations, they were able to get into SolarWinds’ networks, and modify that build process.

      And what’s interesting and notable about this, is, they didn’t go in and modify the source code. What they did is — think about if you’re on an assembly line, and someone made a change like early on, and they all put it together — they actually waited til the very end, the very last step of compiling this package to make this software. It goes out. And they monitored it, they watched it, they looked at it, they learned, they tested. And they ended up compiling in a backdoor — which would give them access to the systems running SolarWinds Orion — for anyone who installed the update, or downloaded it freshly since they did this.

      So, they were able to modify SolarWinds and push out this update to organizations all around the world. And basically, they’d create a shopping list and selectively target who it was that they wanted to go into, and basically break into and further their access. They could look through and see, “Oh, this company, or this government agency, I’m very interested in them.” They could actually activate and walk right into their network, and they’re already sitting in and going into a very sensitive part of that network.

      So in short, it’s what’s called a “supply-chain compromise,” where, they’re really in the build process. Insert themselves to the backdoor into this legitimate software and expand their access — and do it very stealthily for many many months — until, you know, FireEye came forward and figured this all out in December 2020.

      Sonal: Right. And just to quickly, even more high-level-context it — this is playing out against the broader landscape of — for many years now, companies have obviously been using various providers of third party and cloud software and services. We’ll delve into this whole notion of a supply-chain hack, what it means, what it means for the future of security.

      But the thing I wanna really pull on from what you said is that this was very unusual because they didn’t go for the source code, they kind of waited for the updates, and then they were very targeted — as opposed to just sort of spray and pray. So, in your assessment of all the hacks that you’ve seen out there — and Joel I wanna hear your thoughts too here — is this really a sophisticated hack? Because obviously, in our show, we not only tease apart what’s hype/what’s real. I often wonder if that word gets thrown about very casually.

      Steven: Yeah, so from our opinion, it’s definitely — this aspect of it — is certainly one of the more sophisticated that we’ve seen. And it’s not necessarily that there aren’t a lot of smart people around the world, good and bad, that couldn’t pull off something similar. It’s, you know (1), the fact that they did; (2), they did it so strategically; and (3), you know, even if they had gone in and modified the source code, people would still be talking about how sophisticated it was. But, they took it up a notch and basically said, “Yeah, we modify this code, or someone’s watching it, or they audit it, or someone’s watching a check-in process.” Basically it went to a system where none of that mattered anymore. And they just kind of bypassed all that and went, like, straight for the jugular, in what would — I would argue a much more difficult way to go about it, but a lot more likely to meet with success and go undetected in that. I think they gambled correctly in this case.

      Joel: I mean, I think with these kinds of operations — and this is ultimately an espionage, you know, nation-state professional-type operation — from my perspective, the duration and the extent to which these things can run undetected is usually the indicator of how sophisticated they are. And so, like, these long running, you know, really successful campaigns that avoid detection really belies like a level of sophistication.

      Because operational security, right — like, covering your tracks — is actually just about as hard as getting in. And so, you know, the fact that they exercised their ability to cover their tracks for so long, to know where to insert in the process, and to lay low, is just indicative of a level of discipline that you don’t necessarily see in a lot of attackers.

      Sonal: Not just get in, but be able to cover their tracks, which is what both of you guys say. And by the way, we’ve only talked about the duration of when the hack was revealed by FireEye, and that it had been you know several months before. Do you guys have specifics on what the latest date-point is, in that timeline?

      Steven: Yeah, at first, essentially what they did was an experiment early on — and this has been posted publicly. The code in SolarWinds Orion was modified in late 2019. Where basically they made some initial modifications, which actually didn’t do anything malicious or put a backdoor, allow any type of access.

      Software went out. And they basically were able to prove, like, “Hey, I succeeded at doing this, it existed, no one noticed anything” — and essentially waited at some point to move on to phase two, which was, “Okay I can get in and go undetected, I can have it build, it all works, stuff makes it into production, no one notices.” And they said, “Okay, well, I’m satisfied with that. Now it’s time to, you know, go for broke and put the actual code in there, and open the floodgates.”

      Sonal: What you just described, Steven, sounds exactly the way a company builds a product. Like, “Hey, we’re gonna test it out. We’re gonna try an experiment, an MVP, a minimum viable product, if you will. Then we’ll, based on that, decide how to deploy it and target it, and blah blah blah.” I mean, I hate to say that, but that’s exactly what you just described sounded like.

      Steven: Yeah, it honestly wouldn’t surprise me if they had done some way of trying to basically clone their development environment too, and probably tested this — I would guess — probably pretty thoroughly before they even <Sonal: wow> ran the tests within their network

      The hackers’ sophisticated methods

      Sonal: So, they were incredibly savvy in certain ways, in terms of how targeted they were, and the choices they made.

      In the Microsoft blog post, one line in particular really struck me. It said that the threat actors were savvy enough to avoid giveaway terminology like backdoor, keylogger, etc. Instead, they gave their tampered code an innocuous name, “Orion Improvement Business Layer,” that would fit right into a marketing brochure. (This is from an Axios post summarizing it.) “The attack’s crucial door-opening exploit was a small chunk of ‘poisoned code’” — which is what Microsoft dubbed it – “all of five lines long or roughly 160 characters.” And then Ina Fried at Axios goes on to comment (which I had to chuckle, even though it’s sad), was, “This could well be the most damage per character yet achieved in the short history of cyber warfare.”

      So, I am curious if you have any thoughts on some of those — honestly quite clever — things that they did, to hide undetected. And any more specifics you could share there. And then we’ll go into the step-by-step in a moment, too.

      Joel: The fact that they’re not naming variables and naming things that are commonly used in attacks is mostly a credit to the existing kind of antivirus and anti-malware industry. You’ve got a lot of tools that are out there that are looking for this stuff. And you would imagine any adversary that’s relatively sophisticated is gonna run their changes through all those tools to make sure they don’t get detected before they deploy it.

      And so, that’s just table stakes for this kind of activity. It doesn’t really show any kind of real sophistication.

      Sonal: Of course, it just depresses me to hear that — and we’ll talk about this at the end, which is what companies and people can do. Because I’m, like, great — the better and better we get, the more and more sophisticated they get, and it just becomes this like never-ending back-and-forth, back-and-forth escalation.

      Joel: Espionage 101.

      Steven: Yeah. To be completely honest, that stuff doesn’t surprise, especially when their job is to, like, blend in as much as possible.

      But I’ll add to one of the things — and make sure that we give credit — some of the analysis of things we’re talking about today are obviously from — a lot of security communities have come together and published a lot of detail, which has been great. But this is one of the other things that they did, is, they actually used an existing config file that is part of SolarWinds Orion, that’s there legitimately — it was there five years ago, it was there two years ago, it’s there right now — but they actually repurposed that exact config file. They created a specific value and said, if this is a three, you shouldn’t beacon it, you’re basically turned off. And they use values in fields within this to then leverage that file that’s already being read and used by the program, to then also inform it on some of what it should do.

      So they use, like, native, existing files and functionality and things that are very innocuous-looking. And then they did a couple of other stuff beyond that, that are pretty stealthy – although they’re not necessarily rocket science, they are very uncommon.

      ~ One of them is the fact that this backdoor, once it’s loaded, it wouldn’t start its beaconing or calling out for this DNS activity (which I know we haven’t explained yet), but basically, the mechanism by which it actually gives that avenue of control back into the system. You have to meet certain criteria before [you can] even, you know, beacon. For example, if you weren’t “domain joined” — meaning you’re less likely to be an actual corporate asset. You’re someone testing it on a computer, you’re a workstation at home. You’re not even gonna pass the sniff test.

      ~ But what they then do is actually set a timer. And so, it might be actually up to two weeks before it actually starts doing anything. I might be under scrutiny from QA, or a build, or someone might be looking at it when they first install it, make sure it’s not malicious — so they actually say, “Hey, I’m just gonna wait two weeks. I’m in this environment, this is for the long haul, I’m not in a rush to immediately get access to these systems.” So that’s an interesting aspect. It’s actually fairly uncommon to see malware that is on any timer of significance, or driven by a specific event that’s likely to happen very soon.

      ~ The other thing that was really interesting: The malware basically would activate when a certain response was given to its query. “Hey, go connect to this domain name,” or “go connect to this website.” And, those domains that they used were actually domains that had expired. One of the telltale signs when you’re looking into malware and things is, like, “Oh, it was just registered last week or last month or earlier today.” So, this would pass that sniff test, all day long. Some of them had five or six years they had existed. It might even have, like, a website. They picked up infrastructure that had a history to it. They actually owned and controlled these domains. They weren’t, like, hacked domains or things like that, where they were using compromised infrastructure. So, just kind of an interesting note on that front.

      Sonal: It’s interesting and, honestly, a little creepy. I got goosebumps while you were talking, because it makes me think of every long game. The patience, and waiting, and stalking — that really skilled predators do. And I don’t mean to glorify it by any means, but I am just sharing that what you just shared in technical terms — it gave me goosebumps, quite literally. I don’t know how you think about it.

      Steven: When we first saw this in July of last year, we had I think three domains that we had seen used in that actual attack. And as we looked into them, we said wow. Like, we kind of noticed it’s just, like, yeah, these things have a real history. You know, what the hell is going on here? And then we found a way to find more of their infrastructure (even if we hadn’t seen it used in the attack), and they all had this in common. Like, we had a way which we could figure out and find some infrastructure from some mistakes that they had made. That’s why in our post we actually were able to provide a lot of indicators. Like, DHS included that in their list and everything.

      But, other than that, each one of the domains we looked into, we just instantly knew at that point — I mean, we already knew we were dealing with an advanced threat actor, but — we were kind of thinking to ourselves, like these guys have really stepped it up a notch. This was actually the third time we had dealt with them in an incident-response engagement. But this was, like, a little bit different than the other two rounds. There’s a number of things that just made it stand out, and that was definitely one of them.

      Sonal: This might be the first a16z Podcast Network show to be optioned for a movie. I’m just gonna say it right here, on air. Joel, anything to add to that before I switch into the detailed step-by-step?

      Joel: I mean, only if Matthew McConaughey plays me. <Sonal laughs> No, I’m just kidding.

      Sonal: I listen to him on the Calm app every other night or so.

      Joel: Yeah no, I mean I think that’s exactly it. Just the level of preparation, and just the long game that these guys are playing.

      You know, this malware stuff is pretty common on the financial crime-ware type side, right, people trying to steal money. But those actors typically register domain names within a day, it’s just all very phish-y and suspicious. But to see someone build these, like, really advanced, large, complicated infrastructures, years ahead of using it — it just belies a real level of sophistication, you don’t really see every day.

      How the attack took place

      Sonal: Okay. So, just to recap for listeners where we are and where we’re going — we’ve covered what happened at a high level, including some of what’s hype/what’s real, and interesting or undercovered in the media.

      You did a great job summarizing, Steven, but let’s now spiral into that a bit deeper and fill in some blanks that you haven’t covered. Both technical details — you mentioned the beacon, DNS — I want all of it. How folks figured things out — so we can then know what the open questions still are, ripple effects and implications, and then more on supply-chain compromises and what we can all do. But I especially want to know the anatomy of how they got access to the emails. But start from the very beginning of the timeline.

      Steven: Yeah, so the story of the SolarWind supply-chain compromise obviously starts with SolarWinds — and that’s probably where some of the question marks are currently, and they might remain that way. They were breached sometime at least as of late 2019, and then ultimately — what came out later in May of 2020 — pushed out an actual backdoored version of their software. A backdoor meaning, a piece of software that shouldn’t be there, that allows this foreign adversary to have control or remote access into these systems. So we’re talking in late May, that happened. From the cases we’ve been involved in and things that have been published publicly, we’re seeing that a lot of the threat activity started in June and July.

      The SolarWind software would send out this DNS query. So, when you want to go to a website, you wanna go to a16z.com, you type that in. There’s a system called DNS, it says, “Hey, where is this located?” A DNS server says, “Oh, it’s located over here.” It’s the basis [through] which kind of you can find things on the internet, so you’re not memorizing these numeric IP addresses. 

      So, the malware — all it did, once it finally activated — it waited between 10 and 14 days before it would start creating these DNS queries — it would do these DNS queries from the SolarWinds Orion server. And those DNS queries contained encoded data. And if you decoded that data, it gave you different information, but one was information about the network that that machine is joined to. So for — in the example of, you know, say, Microsoft, it might show Microsoft.com or Microsoft.internal. Or, you know, one of these government agencies, it might say treas.gov.

      But it would give this indicator, so that the attackers could actually see who these victims were — because remember, they were indiscriminately pushing out this software, potentially tens of thousands machines. That is an untenable thing to manage, and go and manually look at everything, and try and actually install software and do something of significance. And their goal is to stay under the radar, and not get caught. And then now they have to decide who it is they want to go after further.

      So, they probably have a shopping list that they started with, and they probably have a new shopping list of things — they’re walking into the grocery store and didn’t even know they wanted that, but now they know they do. And they essentially issued commands, and allowed them to initiate this backdoor on who it was that they wanted to attack. And they did this through a specific DNS response called a C-name value. So, it says, “Hey, where’s this host name?” It responds back. They would actually send a specific response to prep it, so that the malware would be waiting to know that next time something happens that it should take a specific action and open the backdoor.

      It would respond with these domains. And these domains would basically be the control points of where the attackers within have the hands-on keyboard — a human is doing this at this point. Someone says, “I am ready to take a look at this system,” and now hackers that are behind this are actually involved, and they’re saying, “Now I wanna look around and figure out. Is this a test machine? Is this a real network I’m interested in? Is this a lab environment? Is this a staging environment?” You know, things like that. And they can figure out, “Is this the real deal? Does this have access where I want? Do I want to proceed?”

      And they did this for — we don’t know how many organizations, and that’s the real scary part in all this, is — you have all these people that have come forward, and they’re, like, big companies or they’re these government agencies, and, that’s just the ones we know about. I don’t think anyone has a real notion of the size and scope of where they took a further interest and then actually did something. In our particular case, we got permission to write up and share details of our incident investigation. The attackers were very focused on getting access to email of specific individuals. So, their goal was maintain access, move around — you know, get what they need — having access to specific individuals, and what they’re writing, who’s sending them, why they’re communicating — was a key focus of what they’re doing. We were able to see that they did that.

      The interesting part, in kind of stepping away slightly from SolarWinds — and why the intel community and law enforcement says it’s likely tied to Russia (APT29, or the Dukes) — we’ve been tracking a group we call Dark Halo, just because we’ve dealt with APT29 on many occasions in the past, but we just have no real way to link the two.

      But what was interesting to us, is the story of this group didn’t start with SolarWinds. We worked three separate incidents involving these SolarWinds attackers, who we called Dark Halo — so, this is a story that starts well before, and has multiple other avenues.

      We had actually dealt with them back in 2019. We had an organization we were doing work with, and we kicked the group out. They went away. In our initial response, we had determined they’d been in that organization for 4-5 years prior. They came back in Q1 2020 through an Exchange control panel vulnerability. You know, mail service — they had a vulnerability that attackers will take advantage of. Got back in, stole email for certain individuals. They were kicked out and removed again. That’s what we did. And then they came back a third time with SolarWinds in July of 2020 again. We didn’t have a good way to prove it, and we took steps and mitigations in place to deal with it.

      So to say, “Hey, how did they get into, you know, SolarWinds,” or wherever else they’re operating — well, this isn’t their only trick. They have a lot of tricks up their sleeve, and they’ve been able to do this and operate for quite some time.

      Sonal: Wait, so how did you make that link across those separate incidents that it was the same group?

      Steven: I’ll tell you, and it was something interesting, is — if we had worked them at three different organizations, we actually wouldn’t have come to the conclusion that this was a single threat group. We wouldn’t have linked the three things.

      Any advanced attacker, anyone in network, they have certain commands and things that they’re gonna do — but they changed enough between each of the attacks, that the actual techniques, the tools — there’s a custom malware, or a commercial script, or a public script, like <inaudible> or a pin-testing framework, or these different toolings, or a web shell — they changed it between each one of the hacks, where it was able to be very non-obvious it’s the same group.

      But what they did is they went after the email of the same people each time — and why we are 100% certain it’s the same group, is — when they would steal email. They would only take a certain amount of email. They would specify, “I want all the emails since the last time I took it.”

      Sonal: Oh, so it’s like incrementally building on the total — oh my God, that’s so fascinating. <Steven: Exactly!> Keep going, yes.

      Steven: So in early 2020 they got back in, and they said, “Okay, well, I want all the email for these particular users since, you know, a specific date in 2019.” And then when they came back in through the SolarWinds vulnerability, they basically said, “Hey, I want every email for these people, and I only want it starting from this specific date range starting in early 2020.”

      So, we had each time they came back and asked for the email since the last time they did it. So in the one case, obviously, they had an intimate and previous knowledge. The other cases we worked, they didn’t have as much knowledge. They had to work their way and kind of figure out the way of the land. So, we’re dealing with the same group in all three incidents — that’s an interesting tidbit.

      Sonal: I was about to say, I still have goosebumps. That’s incredible. That was so good, Steven.

      How the hackers covered their tracks

      Joel: Pretty impressive analysis and work there.

      The things that really jump out to me is, this is something that is linked together over a 4-plus year campaign, trying to maintain persistent access to the communications of high-value individuals.

      I think the other thing that really jumps out to me is that they have a big data problem. They got access to tens of thousands of computers, and potentially thousands of organizations. It sounds like the kind of analysis that Steven has done is pretty unique. There aren’t a whole lot of people in the world that can do that sort of thing. And so, this is probably an incident that we’ll be continuing to understand for the coming months, if not maybe years.

      There’s probably gonna be a really long tail on that. These people are still out there, they’re still operating. What are they doing now? That’s particularly concerning.

      Sonal: It’s interesting because Martin Casado — you know, our general partner, who’s also a security expert — he mentioned to me that he thinks it’s super interesting how interactive the attackers are during the attack. Because it’s obviously a very sophisticated team of people gathering data and making chess moves in real time.

      And it’s so fascinating because when we report and talk about and communicate these types of attacks, we kind of make it seem like it’s malware that does all the work — but it’s really the people that are at the center of it. And then on the other side of it, you have this whole interesting dance, on your end — as sort of this forensics expert with your team, going in, and trying to figure it out, and the puzzles, and everything involved.

      Joel: Well, you know, I heard chess is popular now.

      Sonal: <laughs> Queen’s Gambit, right.

      Joel: This is exactly like playing a game of chess. The difference is that you don’t see the moves immediately — they get revealed over time, and then you’re left kind of piecing other things together.

      Sonal: That’s exactly the analogy.

      Steven: Yeah, I definitely agree — that their goal was to actually not have their moves — what they did never be understood. You know, we noticed the versions of their software that were downloaded. There was an update to SolarWinds Orion — I believe it was in August of 2020 — and that version wasn’t backdoored anymore. It didn’t have the malicious code. So we initially speculated, “Oh, did the bad guys remove it? Did SolarWinds find it? Did it inadvertently get removed?” We didn’t know how it was going down at the time.

      So, they removed the code. They got in, got all this access, and basically said I’m gonna try and remove this now and, like, fly under the radar. So, if they had their way, they would have pulled off like the perfect caper, done all this stuff — no one would have known how it happened. And then the Orion product, basically, would have nothing malicious in it. <Sonal: wow> So, just a, kind of like an interesting other thing that they did.

      Sonal: It is. It’s a very vivid contrast to the analogy of chess, especially given the popularity of Queen’s Gambit, when you see them recording their moves, and the spectators watching — it’s a real contrast to this idea that you’re literally making the move, peeling it back, making the move, peeling it back — it’s really stunning.

      Open questions for the experts

      Okay. So my next question before we talk about some things we can expect to see moving forward — what are some of the open questions still on the table? Like, we know SolarWinds was compromised, but the big open question there is obviously we don’t know how. Then the second big thing in the Microsoft post that I saw (and Steven Sinofsky pointed this out), which is, you know, they do this outline, but we still don’t know how the signed code was signed, so that whole idea of “sign the code” is a bit of a mystery still.

      I want to hear from you guys, what are your open questions — or what are the open questions the industry is still looking at, or that people should or shouldn’t look at?

      Steven: Sure, yeah. So, how was SolarWinds compromised? Obviously one of the open questions. You could spend as much time and resources — you could use infinite resources, and you may not ever be able to answer that question because that system is gone. It was wiped. All the logs are here, that was never logged, or it happened five years ago. So, I would say the scariest part of this — people are finding out about this in December, for something that was operationally live in May. They had a looong headway into breaking into different organizations, doing that shopping list. And there are going to be — and there have been — from this very group, and as a result of the SolarWinds compromise, more supply-chain breaches.

      Some people are breathing a sigh of relief, “Ahh! I didn’t run, you know, SolarWinds Orion software. I’m safe.” That’s not necessarily true. We’re not trying to sow fear, uncertainty, and doubt that everything is untrusted — which arguably, you need to go to a typewriter, send pigeons now — but it’s IT companies, it’s security companies, it’s managed service providers, it’s managed security service provider. There’s these different people that were running SolarWinds that then had this level of access to either directly get into networks, get into email, get into authentication systems to provide software or software updates or software downloads. They 100% certain had access to numerous networks and systems that would allow them to rinse-and-repeat SolarWinds, probably on numerous different scales, in numerous different ways. It doesn’t have to be through a build-time compile. It could be, they change a download, they change an update process. They took keys, or secrets, or remote access protocols, or passwords that got them into like other networks or other systems.

      So, the scary part is, is that the supply-chain compromise here is just causing a chain reaction that’s probably already impacting other organizations that have no idea. I think that’s one of the biggest questions, is — who else was victimized that we don’t know about, and what do they do?

      Sonal: So what you’re basically describing is, like, this complex, adaptive system — like, everyone sort of networked and connected trying to tease apart the scope and ripples of this is gonna take ages. And we might never, ever get to the bottom of all of that, because of that connectivity.

      It’s interesting because General Paul Nakasone, or Nakasone — I’m not quite sure how to pronounce it — he heads both the NSA, the National Security Agency, and the military’s U.S. Cyber Command. One of the things that they talked about is that developing a coherent, unified picture — what you just described, Steven, of the extent of the breaches — has been difficult. The challenge is that, “He’s expected to know how all the dots are connected, but he doesn’t know how many dots there are, or where they all are” — which is kind of a distillation of what you just described. What are the other open questions that are on the table?

      Joel: For me, the big open question — and with all of these really sophisticated breaches, the first is, how many stupid things led up to this? Like, how many ridiculously, easy-to-solve problems, like applying security patches, or using two-factor authentication — like, how many of those kinds of things we know we should always do are responsible for this — is always front of mind when we see this.

      Because I think when you double-click on these, a lot of the times it starts off in a fairly innocuous way, which is, like, someone guessed an account, or someone got access to some account. But as this event shows you, if you give a sophisticated actor a toehold in your organization, they’re just gonna run through it. So, that’s the first one.

      And then the second one is, we think of these breaches — because of just the way the media covers them, and the fact that they kind of show up sporadically — we think of them as, like, events in time that have a start and finish. But, in reality, these groups are still running, and we’re still facing them. You don’t know the implications of any of this stuff for a while. Like, you don’t know if they were getting into the Department of Energy to read, you know, Rick Perry’s old emails, or if they were getting in there to steal futuristic bomb designs. Maybe there’s gonna be some new weapon that pops up in 15 years and it’s, like, linked to this breach. And we’ve seen from these breaches — like, if you go all the way back to some of the first ones that have been publicly reported — you know we’ve often seen that the goal of these is either to spy on individuals and get some intelligence there, or to steal the designs for things that people want to go recreate.

      Sonal: Right. And don’t forget that oftentimes — I think we often forget to talk about, when we talk about intelligence — it’s often in the form of blackmail, right? Like, we’re not just talking about stealing IP and obvious secrets.

      Because a lot of people dismiss this as, “Oh, email. I just book events and share, like, photos with the family in my email.” I don’t think they realize that it’s such a vector to all these ways of really exposing who you are. It’s your identity, in many ways. So, that’s another way to think about that too.

      Joel: Absolutely.

      Sonal: Anything else on the open question side?

      Joel: So, a bunch of other secondary breaches are now being reported on. Some of the Microsoft stuff, you saw that there were people creating reseller accounts, or trying to get reseller access to people’s Office 365 enterprises. And then there were certificates that were compromised for things like Mimecast, and maybe perhaps other services that are out there.

      And so, like, this picture starts to emerge that there’s these — lots of fires just started burning. And it’s always really difficult to tell if it’s one fire massing together, or just a bunch of different people that are acting independently.

      Sonal: That’s actually something I wanted to really quickly touch on before we go into the rest of this. Because the thing that was confusing to me is, okay — so, I read the Microsoft post. You know, like, there’s some intrusions. That there was a partner for Microsoft, actually, that handles cloud access services. We don’t know how connected or not connected it is. Then you have a reseller gaining access to Microsoft customers’ Azure accounts. Then you have this reported Russian state-sponsored effort exploiting a VMware flaw that the NSA warned about last month, that takes advantage of a recently announced vulnerability in VMware Workspace One access. Access Connector, Identity Manager, etc. And, this is according to the NSA, that they’ve had at least one case — that they’ve successfully accessed protective systems by exploiting the flaw.

      And then you have, like, you know, one after another, and they issued a patch. I mean, I am reading all these at the same time and I’m like, is it all the same thing or not? I think that’s what you’re saying, Joel, about — we don’t know if it’s all one fire, or a bunch of fires. And do you guys have any thoughts on how to connect those dots, if at all?

      Steven: So, as a general statement, I would say what we know about this attacker that we call Dark Halo — the people behind the SolarWinds hacker — they’re extremely adept in methods that allow them to gain access to email or systems involved with email. So, things like trying to get access to an Office 365 or Azure AD environment through a partner organization. Or by stealing some, you know, SAML tokens or some kind of authentication mechanism. Or, trying to get access through some other — possibly through a vendor — to get access to that same data to email data, essentially by any means necessary. I would say all of those are very on par with what we’ve seen this attacker do and focus on, and what others have seen. A very good chance that they are related.

      But even if they weren’t, it just kind of underscores that there’s a lot of people trying to get access to this data. And now you need to focus a lot more on the cloud, on the technologies that are used to secure the cloud or that have access into it. And the things and places where people don’t always look — because it’s new to them, or they never looked at it, or they didn’t know to look at it — so, I think this event will actually end up advancing security in many ways, because it’s causing people to think about and do things that they weren’t realizing before. And as you can see, the bar’s been set higher to where they can’t walk right in the front door anymore, right? They’re not easily able to get right into these organizations by compromising, you know, the core network or the system administrator and the other ways which you could get there.

      So, in some ways, it’s a sign that security has improved a lot — but also that there’s a massive amount of work to do at the same time.

      Sonal: It makes me again think of the chess analogy, and when you have a player that comes to the table that has a set of moves — like, patterns that are well beyond what the human mind can even comprehend — and that makes me think a little bit of even, like, AlphaGo playing Go with a real chess player in Korea. And how you know the system made moves that they considered very alien, but that a human being would never have done, but that still follow the rules of the game — the constraints of the game, that is — and yet were completely novel. And if you just keep seeing more and more moves kind of grow and become more and more sophisticated on both sides — even as we may improve, like, there are gonna be alien moves at some point.

      Steven: Well, to be completely honest, they’re undoubtedly highly skilled and disciplined — which, if you think about it — okay, if we go back to the chess analogy. You know, are they a master, are they a grandmaster? In some ways you can say, okay, they’re a grandmaster — but most of their opponents are unranked. So, they have this kind of lower skill, and their strategy is easier. But then they’ve been able to go to these people who maybe their security defenses are much higher ranked, and they’re using that skill set, that knowledge, and that kind of cat-and-mouse, to still get into those organizations. But to have to do that, it shows that people have leveled up quite a bit — which is a good thing for these companies and the security industry.

      But at the end of the day, they still managed to either capture that king or get them to knock it down. I guess no one’s really thrown in the towel. No one has surrendered, that I’ve seen so far. But, I would say they’re winning a lot of matches, and they’re playing a lot of them simultaneously.

      Sonal: Right. But they are not (to be clear, to your point), an alien player, like an AlphaGo. They’re still moves that are human, just very skilled.

      Steven: At least from what we’ve seen, but who knows, like, what we’re missing though, right?

      Broader security trends

      Sonal: Right. Okay, so now the big picture questions. We’ve covered what happened, how it happened, the details. We talked about this, you know, phenomenon of supply chain attacks, chain-of-chains, what it means. I would love to hear what you think about this, when you think about the broader trends at play.

      Joel: Yeah, absolutely. I think on the podcast several times — I know I sound a bit like a broken record — but we’ve talked about the biggest challenge being securing the supply chain. And how all these businesses that are becoming software businesses are actually becoming reliant on other people’s software. And so, it’s not just a matter of the stuff that you write to run your company, it’s also the matter of the stuff that your suppliers are writing.

      And as everyone knows, security is really difficult, and it’s hard to secure your own things — and then having to worry about the security of your suppliers is adding an additional layer of complexity.

      And so, over the last couple of years, there’s been a lot of investment in trying to understand third-party risk management, vendor-risk management. How to glue these things together. There are several different approaches, everything from private systems that will look for vulnerabilities and report on the risk. There are publicly available standards, different trade groups are trying to develop their own standards for security — and then certain vendors are trying to come up with their own standards. There is no easy answer, and so what you’ve got is a lot of different approaches that are being tried, and a lot of experimentation that’s taking place. This is probably the first breach at such a size, scale, and scope. So this is kind of the watershed moment for that third-party risk management.

      And there’s any number of other suppliers that are out there that are in very similar positions, right, and it could be a company like SolarWinds, or it could be an open-source repository that a bunch of people are building into their applications. There are any number of different ways.

      The thing that’s really difficult for me — based on where I sit and what I see — is if you play through all the different potential solutions that are out there, it’s really hard to know which one of them would have actually prevented this? So, like, if I went to any of SolarWinds’ customers and said, “Hey, what’s your vendor risk-review report on SolarWinds?” You know, before the breach, I’m sure they would have said it was a wonderful company, it was doing everything, they passed our review, they answered our questionnaire. You know, they’ve got the people hired, they have a program. And so, it really comes down to how do you actually measure these things, and how do you measure the risk in that third party, and how do you effectively mitigate against it?

      Steven: The third-party risk or the vendor-risk management or how that someone evaluates this — it can only go so far, right? Like, how would you evaluate SolarWinds and the Orion product any differently than you would Microsoft Windows and Defender and how it updates and things like that, right? So there’s limitations to what you can do. I mean, you could audit them, or find out their code-review process and all that stuff — and they could have passed that all with flying colors. Or does your checklist say, “Are you looking for advanced adversaries, you know, injecting themselves into your build process at the highest levels of sophistication and espionage?” But even if they check yes to that, which they might not, they probably aren’t having an effective way or mechanism to do that.

      Sonal: One of the things that Alex Stamos — people tend to over-quote him, but he did have a good tweet about this, which is — “There is no good reason for most enterprise software products to talk to random internet hosts all day. It might be time to move on to an outbound network-permission model for Windows servers, so connections only allowed to domains and signed manifest plus internet as defined in GPO.” Is that the right thing to do? Should people be air-gapping? Like what should people be doing?

      Steven: We deal with sophisticated breaches all the time, and this can even apply for, like, crimeware and other stuff, but that is a recommendation that Volexity has been giving for years and years and years to organizations. And it’s often in an incident that we say, “Hey, your domain controller, for example, doesn’t need to be able to talk to the internet.” There’s obviously exceptions to the rules and everything, but usually those can be defined, especially with next-generation firewalls or modern firewalls — you can define what is actually needed, and allow them to do those things, and not allow them to do anything they’re not explicitly required [to do].

      And that’s a model that is the least privileged, it’s like the least-access type model. That’s a little bit harder, depending on your organization, to enforce for users and workstations where they need to browse the web and do all this stuff. And that’s what content filters and certain restrictions are for. You know, unless you’re into, like, a DOD environment or something where it’s a lot more locked down. But that’s usually accepted in a lot of, like, commercial organizations.

      And a server is where — an attacker, if they’re gonna install malware and do things — usually go for it, because that’s where the supply chain, that’s one of the big areas to get to it. Or those are the machines that are not at home, or requiring a VPN. They’re always on, you know, they don’t get rebooted frequently. That’s where malware gets installed a lot, because it’s something that they can count on, and it’s regular. Being able to prevent that, and limit what those can do — that model, if that had been put in place for organizations with SolarWinds — in this specific instance, it would have mitigated that threat.

      Now, if I start thinking outside the box, and this attacker used DNS — but what if they had done command and control activity, and issued commands, and had done that all over DNS? So, the SolarWinds server talks to its local DNS server, your local DNS server goes out to the internet. If they had modified this malware and actually did all the command and control over DNS, instead of doing it over this connection, that paradigm and that shift would have been a lot more difficult to mitigate.

      But that’s the type of issue and security item we need to think about. You could proactively try to address that, or just say, “Hey that’s a lower likelihood, and I’ll address it if that happens.” But by and large, it’s a best practice with regards to minimal access, specifically for servers connecting to the internet and different resources.

      Joel: It’s funny talking about this, because it’s like the history of the security industry is the history of unreasonable requests.

      I know that a lot of people are jumping up and down talking about, like, don’t let production talk directly to the internet. And if you worked at a bank you know for the last 20 years, that’s been the case, right. Like, highly-regulated industries, and people that have invested heavily on security, have always focused on doing these rather idiosyncratic things that don’t make a lot of sense — but made a lot of sense to people who either come from an incident-response or a deep security background.

      You know, back in the 90s, I remember being involved in strenuous debates about why you need to encrypt traffic moving within your data center. And everyone thought it was the most asinine thing because it’s a private link. You’ve got MPLS, no one’s gonna listen to you — and then Snowden released his documents. And it became really obvious why you want to encrypt your data within your data center.

      So, this is just another example where people have been giving best-practice advice, saying, “Hey, you need to make sure that random servers, random production systems, can’t just talk arbitrarily to the internet.” And the response to that has generally been well, that’s an unreasonable request, that takes a lot of work, I don’t know that we necessarily wanna do it. And, there was never a particularly great reason or piece of evidence to point you to say “well this is why”. So, this is why — why you wanna limit that access. And there’s probably a list of other things that are equally unreasonable requests that security people would ask you to do, and eventually they’re gonna have their “this is why” moment.

      Steven: Something that Joel mentioned earlier, which I think is really important, is — a lot of organizations aren’t doing blocking and tackling. They don’t have two-factor authentication on the remote access to their network. They’re using weak passwords, they’re not patching. They don’t know where their assets even are, and their build process is not secured. They don’t even do code auditing or check-in their code. I mean there’s a lot of low-hanging fruit for most organizations. They haven’t even been able to kind of get into some of the basics.

      But I think a big problem that a lot of organizations — whether that’s a government, commercial organization, or really anyone, whether they’re a small company or these massive companies with huge budgets — a problem that they’re facing is that if you had certain security data, you could immediately and very easily answer, “Did I have a problem?” One, did I run up vulnerable software? Because maybe, you patched. You know, I don’t know. Maybe I never ran, and I skipped a version. If you had all your DNS queries logged and the responses, you would say, did I get a C-name? Did I even call out to that command and control activity? There’re certain logs from the endpoints that SolarWinds has instrumented in these event-log data. If you had been capturing that data, you could answer that question.

      Sonal: Most companies do capture that data, don’t they?

      Steven: It depends. If you went into SMBs and mid-sized businesses, even some large businesses, I would say a lot of them aren’t actually logging or keeping DNS logs. And if they are keeping DNS data, it may not be query-and-response. And event logs — the vast majority of organizations don’t have a centralized and long-running retention policy for event logs. But even if they do, their data retention of how long they were keeping this data did not go back far enough.

      They actually had data — they have data going back 30 days, they have data back 60 days, 90 days — so they’re finding out in December about a breach inside of activity that happened and then potentially initiated in May. And, “Oh, I kept all this great data, but I can only go back three months.” Three months from December, it’s September. And for a breach that happened in June or July, that’s, in some respects, useless. That’s a scary place to be in, to not know if you were compromised, or if you were when it started, or what happened, or where did they go, how did they pivot. It’s a missed opportunity, and probably a bit scary for some of these companies is that I was collecting all the right data, but I didn’t have it for long enough, so I don’t actually know.

      Sonal: Wow.

      Steven: We’re helping a lot of companies right now to see what resources they have. You know, we specialize in memory forensics. We’re acquiring memory from their SolarWinds server, acquiring disk artifacts, or full disk images, you know, any log sources. And we have some stuff that we can potentially go in and say “doesn’t look like it” or “definitely, yes you were.” You know, we see these items that clearly indicate that you got a second-stage breach, and you need to expand this out. But we can’t give anyone, if they’re on limited data, a confirmed clean bill of health.

      Sonal: It’s a little bit like going to the doctor and having, like, maybe a continuous glucose monitor for the last year — but you only have the data for the last three weeks stored. And it’s sort of like, “Okay, here’s what’s happening. I’m getting sick, but I only have the three weeks.” It’s just, like, a really tough thing to figure out.

      Advice for businesses and consumers

      I wanna break this down by advice for big companies — like, large enterprises — advice for small and medium-sized businesses, and advice for consumers. So, let’s start with the big companies, because the best threat actors, they understand the reality of modern enterprise IT. What are pieces of advice — or mindsets, even — that you have to offer for how chief security officers, CEOs, leaders should be thinking about the implications of this for their business?

      Joel: I mean, I’ve spent a lot of my career in big companies, and I think the thing to do right now is to think about strategy. Like, the tactics are great, and there’s gonna be a lot of people chasing a lot of actions over the next days, weeks, months. But I think the strategic view of how an organization wants to think about security — as we start to understand what happened, and how it happened — we’ll consistently see in some organizations that security either wasn’t funded, it wasn’t empowered, it didn’t have a remit to act. It may have been under assault. People often view security as being a cost center, as something that you know contributes to the lack of performance in a business. And that is an attitude that is still quite popular.

      So, I would say that, like, it’s really gonna be about figuring out strategically, where does security sit, what’s the right amount to spend on it, how do you effectively empower it, and then how do you partner and build security into your business so that it’s something that helps enable it, versus something that holds it back.

      Steven: Yeah. Generally, no one really thinks like security is not important. I don’t think we ever hear that. Now, action may speak louder than words sometimes. But I think a lot of people think about, “Oh, it’s an afterthought. I’m gonna add it later,” or “Oh, yeah, yeah, well, you know, we’ll do that one day.”

      And I think, like, our main advice to a lot of these different organizations — whether it’s a startup or a midsize company, a company that’s growing really rapidly – is not necessarily that they need to come out of the gate and have to have every imaginable security product, they need to be auditing all their source code on day one, they need to have everything locked down, and the latest firewalls, and this filter and all these EDR products. But it’s like, think about that stuff. Are you doing the two-factor? Are you lazy, like, “Ah, I don’t need to put, you know, two-factor on my Salesforce account where all my most sensitive contacts and information is in my organization.” Or, “Ahh, I don’t really need to put it on email. It’s like, it’s easier if everyone can just log straight in.” Or, “I’m just gonna share this root, you know, Amazon key to get into AWS, because that’s just how our organization’s growing, and we’re not formal.” There’s things that people can do — best practices, actions that organizations can take — see what you can do now, see what you can do along the way, and put that on your radar, so you’re not in a position where you’re starting from scratch, or trying to investigate a breach, or figure out if you even had a breach. We all knew [what] we should have done, and we knew that two years ago. And we run into that, a lot.

      Sonal: Don’t wait till later. And now advice for advice for consumers, like, just day-to-day people like family members, etc. What would your advice be for how to think about things like this?

      Joel: We wrote a really excellent blog post last year called the “16 Things You Can Do to Protect Yourself,” and I would strongly recommend that people do all of those 16 things. It’s all really basic stuff, and it starts with two-factor authentication, patching your systems, and goes all the way down to how you want to think about securing your potential social media accounts, etc. So.

      Steven: Yeah, we issued some guidance, and it’s a couple of intersections of prevention and detection, and then remediation, if you have an actual threat or concern.

      From the prevention side, prevent unnecessary access from your servers — like, your SolarWind server, other devices — from talking to the internet. That’s a prevention mechanism. You know, monitor your assets, see where they’re logging in from, if you have that centralized logging or like a SIM, same thing. Make sure you’re capturing either from event logging or your endpoint security products or that the actual commands being run on the system are being logged. Because that can be pivotal and be critical to 1) detection — but even if you’re not actively monitoring it, you can go back and say, “Hey, what commands are running on this server that’s not consistent with what our system-admin or the typical activity would do.”

      But take a look at your mail server, look at where your email is going, because that’s where the attackers, I believe — they’re way ahead of the game with regards to the things that they can do in Office 365 and Azure AD, where they are so familiar with the administrative commands and what to do from a sys-admin aspect. They’re able to do a bunch of things and hide in ways that people have never even thought about and encountered. And it’s not necessarily, like, they’re ghosts or they can’t be found, people just don’t know to even look for it.

      And then, just from a general remediation perspective — once a device has been backdoored or compromised, it’s an untrusted system now. Don’t just, like, roll back to an earlier version, or, I’m just gonna upgrade to the new version. We say, hey — blow that whole system away. Start with a fresh, clean install. If you’re putting SolarWinds Orion back on it, download the newest version that’s not backdoored, and start everything from scratch.

      Anything you used on that server, if your SolarWinds set up for the Orion had credentials, change all those passwords, and make sure those passwords aren’t similar to, like, old passwords that were used. You know, another thing, too, is — any sensitive API key integrations and things — like, we saw two-factor bypass to get into email by this threat actor. Because they had taken a secret key, and would generate cookies and skip into the email system while not actually being challenged for two factor.

      You’ve got to think about the stuff that someone could steal if they’re in your network, related to this — but also that advice extends well beyond this threat actor and SolarWinds specifically.

      Sonal: That’s great. I’ll include links to Volexity’s blog posts as well as the “16 Things That You Can Do to Secure Yourself” in the show notes. Bottom-line it for me — what’s your takeaway?

      Joel: It’s consistent with what we’ve been saying for a while now. The hardest problem to solve is third-party risk, and this is probably the most significant third-party breach that we’ve seen in history. And so, I think it’s gonna take us months to really understand what happened, and probably years to fix it.

      Sonal: Thank you so much, you guys, for joining this episode of 16 Minutes, which is a 3X 16 minutes.

      Steven: Definitely, thanks for having me.

      Joel: Yeah, thank you so much. And, Steven, it seems that we’re always catching up when the world is burning down.

      • Steven Adair

      • Joel de la Garza is an operating partner at a16z focused on information security related companies. Prior to joining the firm, he held top security roles at Box, Citigroup, and Deutsche Bank.

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      Developers as Creatives

      Jeff Lawson, David Ulevitch, and Sonal Chokshi

      The rise of developers — as buyers, as influencers, as a creative class — is a direct result of “software eating the world”, and of key shifts in IT from on-prem to cloud & SaaS to the API economy, where application programming interfaces are essentially building blocks for innovation. Developers therefore not only play an outsized role in high-performing tech companies — but managing and motivating them is actually critical in ALL companies, since every company is a tech company (whether they know it or not).

      As every industry turns digital, and a company’s interface to their customers IS software, “asking” one’s developer is the key to solving business problems and to thriving not just surviving, argues Jeff Lawson, CEO and co-founder of cloud communications platform-as-a-service company Twilio, in his new book, Ask Your Developer: How to Harness the Power of Software Developers and Win in the 21st Century. So in this episode of the a16z Podcast in conversation with Sonal Chokshi and David Ulevitch (who previously argued “the developer’s way” is the future of work), Lawson shares hard-earned lessons learned, mindsets, strategies, and tactics — from “build vs. buy” to “build vs. die”, to the art and science of small teams (“mitosis”) — for leaders and companies of all sizes.

      But what does it mean to truly treat developers as creatives within an organization? What does it mean to be “developer first”? And how does this affect customers, product, go-to-market? All this and more in this episode.

      Show Notes

      • How software has become an essential part of the supply chain [1:40]
      • Discussion of whether companies should build their own software or buy it [4:10]
      • Including developers in the sales process [8:10], seeing them as influencers within a company [14:08], and having a culture of asking developers for input [17:13]
      • Defining the “developer way,” or the culture of work that developers prefer [20:00]
      • The importance of small teams [23:47] and how to keep teams small as a company grows [28:37]
      • Lessons from Twilio’s IPO and expanding the role of sales [31:17]


      The new software supply chain

      Sonal: Since I hate starting with the “why’d you write the book” question, I’d love to instead start with the big picture. We’ve seen 3 eras of software: from on-prem to cloud and SaaS to the API economy; you not only outline these shifts in the book, but you go further and argue that software is the new supply chain. So, I’d love to hear more about how you think about that to start us off.

      Jeff: Yeah, thank you for asking, Sonal. I mean, it’s interesting, I come from Detroit, it’s the automotive capital of the world. I grew up around cars, and so many people that I knew, if they didn’t work directly for the automaker, they worked for a company that supplied the automakers. And it was very easy to understand the very sophisticated supply chain that allowed them to manufacture a complex thing like an automobile: General Motors doesn’t have to manufacture every little piece of that car. There’s companies who specialize in speedometers, and headlights, and seats, and all this kind of stuff — they just pick the vendors that are going to help them get to market as quickly as possible, with the best product.

      And that’s what finally the software industry is developing. Software’s obviously been around in some form or another for 50 years, and in its modern internet-enabled form for close to 30 years. I was a developer starting in the 90s, writing code; back then, you basically had to write it all yourself, and you could pull in some open source stuff here and there. But building software has gotten easier and easier and easier, because of sophisticated APIs, and abstractions, and all this kind of stuff.

      However, the scale of the internet, and building applications at internet scale, has gotten so much harder. And so, if you think about the progress that’s happened over the last decade or so of APIs that run in the cloud — if you need a piece of functionality, whether it is compute storage, payments, communications, maps, you name it — all you need to do is sign up, put in a credit card, and plug in this few lines of code, and now your app is supercharged with these powers… What that really is, is the development of a supply chain for building software.

      Sonal: That’s great. And so, I mean, you’re basically saying software is innovation in that context. And we totally agree, obviously. The obvious next question that begs is, what do you build versus buy? In your book, you had a really neat rule of thumb, which is, that software that faces your customers, you should build; anything where your customers will be saying, why doesn’t it do X, and your answer is, well the thing we bought doesn’t do X, et cetera, et cetera — You basically argue that you can’t buy differentiation, you can only build it.

      So, talk to me a little bit more about that, and really, where do people then compete? Because if everyone has access to the same APIs, like, where’s the differentiation?

      Whether to “build or buy?”

      Jeff: Absolutely. You know, something happened over the last 15 years, which was software went from the back office to the front office. It went from being something customers don’t care about to something they experience every day.

      David: The remote control in our pocket — it’s how we interface with the companies we do business with!

      Jeff: You know what I called my iPhone for a while, I called it my “summoner”.

      David: Yeah, exactly.

      Jeff: Think about your bank. Twenty years ago, your bank was a storefront that you walked into; it was clean, the teller was friendly, and they gave your kid a lollipop — okay, I like my bank. And now your bank, of course, is a mobile app. Suddenly, the interface you put in front of that customer is the perception of your product and of your value as a company. You like your bank if the mobile app is fast; if it is bug free; and if it has a lot of features and functionality to make your life a little bit easier.

      Back in the days when it was back office, it would be common for IT departments to say, “Okay, should we build versus buy?” And a vendor would inevitably come in and say, “Don’t reinvent the wheel”, and you just bought something off the shelf. But now in a world where the software you use is your source of competitive differentiation, the act of building is the act of listening to your customer — and so now, the question has gone from build versus buy to build versus die!

      Because one participant in that market starts listening to customers, and using the agility of software to innovate faster and faster and faster. And then the incumbents in that industry start listening, and they say, “Oh wow, we’ve got to do the same thing” and so they start becoming builders of software as well — this Darwinian evolution is going on in every industry. And so the buy act is one enabling you to be the best builder in your industry. That’s how I think about it.

      David: That is great.

      So, you know, if you are a company, you’re thinking about, “Hey, I do want to start building out developers; I want to have teams. But do I just use all the AWS kind of APIs? Do I go to Microsoft and use Azure?” There’s all these new startups that have APIs, how do I think about those things, how do I make those decisions?

      It seems like AWS is creating an equal API for every startup that’s out there… how do you talk to business executives about those decisions?

      Jeff: Well, it’s like any competitive dynamic in any supply chain. Which is, you want to pick the vendors and the partners who are going to enable you to go build as fast as you can. Every company has their areas of strength, their areas of expansion. And APIs offer you the ability to pick the services that are going to serve you best.

      And one of the things I talk about in the book, is actually trusting your developers to help you navigate the vendor choices that you have. I remember back to the early days, when I was a developer, I would do a Google search; oh, this looks good; okay, how do I get started? And you click and it would say, “Contact sales. And, you know, if you sign an NDA, you can read the documentation.” You’d be like, okay, back, never mind, I don’t want that.

      Because for a developer, documentation is the ultimate marketing. Yes, every company has a marketing website that’s pretty and hand-wavy. But at the end of the day, the documentation of an API is the perfect description of what that product does — it literally describes every in and out of what the product does. And you don’t have to believe a salesperson, you don’t have to look at a slide deck; which is like faster than in the old world, you couldn’t even get a meeting with the salesperson, right?

      That’s why they are becoming so influential in adoption cycles and sales cycles, because a developer can — for free — read the documentation, make an evaluation of what they think the product is. And then for literally dollars, build the prototype, and test those hypotheses, and put it in front of customers, and actually do a beta. And that is a completely revolutionary way to de-risk these projects and take them from a bunch of hypotheticals — with a lot of budget and energy put into signing contracts with vendors and taking meetings — to actually just getting hands on a keyboard, building the thing, and putting it in front of customers.

      David: That empowered developer really transforms the go-to-market for API companies, right? I mean it changes the way you do customer success, and the way you do onboarding, if they’re going to build that prototype before they maybe even talk to you — it must really radically transform the way you think about what an enterprise go-to-market organization looks like in an API world.

      Seeing developers as customers

      Jeff: I’ll tell you a true story; WhatsApp is a very large customer of Twilio and has been for a long time. And like, literally, that is a Yahoo email address — of Jan, signing up for a Twilio account back in 2011 or ’12, or whenever it was — and this is one of the big differences between API-economy companies, and other companies, who say they serve developers. At other companies — who regularly launch APIs and say hey, we’ve got a platform — the developers are a strategy. At API economy companies, developers aren’t a strategy… They’re our customer. They are our revenue. You’re never going to pull the rug out from under them, because you are dependent on them for the health of your company. And that’s a very different world than other companies where the customer is an advertiser or somebody else; for the API economy, you have to treat the developer as your customer.

      David: Yeah, the example of the WhatsApp story of having an individual developer sign up, means that you really rethink marketing, communications, how you engage with those customers, how you measure the metrics, how they’re using the product.

      Lots of companies don’t have great visibility into how their customers are using their product. But by definition, an API company has incredible visibility into how their product’s being used. And that has never been possible before. You know the nice thing about an API company is you don’t have to track: Did they build a prototype? Are they going into production? Are they making one or two calls a week, are they now making thousands of calls a week? Maybe we should reach out, see what they need, what features are missing, have a product manager engage them, and you know, keep those people close to the customer, whether it’s developers or product managers.

      And so the order of operations of the traditional enterprise go-to-market HAS shifted. What used to be a whole bunch of pre-sales, marketing material, and brochures, and websites; now, as Jeff said, it’s the documentation. But then after that, you do want to come in with that white-glove kind of a service and really embrace your customer, understand what their needs are, understand what the opportunities are; you know, maybe rethink your roadmap and all these things, based off of how people are using the products.

      And I think that creates lots of other opportunities for startups to actually support this new kind of a go-to-market motion.

      Sonal: I think the most under-discussed, but most important aspects of this conversation IS this notion of keeping developers close to customers. That’s a really novel idea for a lot of traditional companies; it’s actually probably even a novel idea for a lot of established software companies, frankly.

      You both mentioned the documentation. But what really struck me, is it forces developers to be better communicators. Because you’re essentially having to explain (even if you don’t write all your own documentation), what is this value, what is this thing you’re doing? And that is another segue to this topic of how does one keep developers close to customers? Does that mean you literally tactically put them in front of the customer; are they now the front interface to customers? Are they taking the customer success calls? Are they taking, you know, reports?

      Like, what does it actually mean to keep your developers close to customers; and, how should this happen (or not happen)?

      Jeff: That’s a great question, Sonal. I think the answer starts with my assertion that being a developer is fundamentally a creative exercise; it’s not merely a technical exercise.

      And I think that’s something that is really misunderstood about software developers. You know there’s this pop culture myth about developers that’s propagated by Hollywood; and, look, there may be some truth to that, but, really, developers are not just, like, calculus, you know, math nerds. In fact, we did a survey of software developers, and we found more than half of them played a musical instrument, and it was like three quarters of them did some sort of artistic thing on the side. And the act of writing software is creative problem-solving.

      But that creative problem-solving skill doesn’t end with writing an algorithm — it really goes all the way to the types of problems that you throw at developers. And so one of my biggest statements in the book is, instead of sharing solutions with developers, share problems. Instead of handing a product-requirements document that was written by some MBAs, and throwing it over the wall, and build it to the spec — you know, having a developer basically be a digital assembly line worker — share the problem with them: Hey, we’re trying to make it so customers can sign up for our product and get productive in 30 seconds instead of the 20 minutes which it takes today. NOW you unlock the ability for that developer to use the full creative energy they have.

      David: You and I both self-identify still as software developers; I still write code and I’m sure you write code as well. The reality is, as you said, these developers are creatives — and like any real creative, they want people to use their work, their art.

      I actually believe that there’s like a selfish reason why people are open source developers, which is that they just get a much wider audience much more quickly. And then inside of a company, you want to know that there are people that are going to pay tens of thousands, or hundreds of thousands, or millions of dollars to use the code that they wrote — and, find that extremely satisfying. One of the greatest tropes that always bothered me was this idea that developers need to be protected from the customer.

      Jeff: Don’t get me wrong; you don’t want your software developers handling every support ticket and every sales cycle. However, if you don’t poke holes in those siloed walls — and you treat them like these precious things that can’t be bothered by such trivial matters, like customers – well then you are doing a huge disservice, ‘cause you’re essentially blinding the developers to why they’re writing the software in the first place.

      And so you need to intentionally poke holes in those walls, and I think product managers are actually the key to this. At a lot of companies, product managers see their jobs as shielding developers; and I think the job of product managers is to figure out how do I facilitate the right interactions between the developers — who I want to be able to have instinctive understandings of my customers and their problems, and the jobs-to-be-done by those customers — and the development team who’s there to solve problems. Because when you have an instinctive understanding of the customer, well so many other ways of solving problems arise, and so many other ways of thinking.

      Developers as internal influencers

      Sonal: It’s super important to treat your creative class that way. And it’s so funny because we also talk about the rise of design a lot; and this is a similar shift that’s happening with designers when it comes to designing technology products as well.

      I’ve noticed that people often do the same thing with writers and editors. Like they give you this specs doc, and I’m just bring ‘em more upstream, like, embed into your flow… ‘cause we’re going to hear things that you don’t know to ask us or tell us.

      Jeff: Sonal, one thing that comes to mind is the parallel between the shift that’s happened because of personal computers and the internet for other creative classes — we’re all aware of the fact that you can use GarageBand, or Pro Tools, and a musician in their own home can record a song with basically the same tooling that the professionals use. And if your music is any good, you can develop an audience of millions of people, as a creative. The same thing for film production or video, right, like you used to have gatekeepers, who were studios, and you needed millions of dollars of equipment to make a movie; now, anyone with an SLR can make a movie, can edit it on Final Cut Pro (the same software that they use in Hollywood), and upload it to YouTube.

      And so people well understand what’s happened to those creative disciplines. But really, the same exact thing has happened for software developers. Which is a software developer can take the same infrastructure that’s used by the largest companies in the world; can build a software app on the internet; and get distribution with Google AdWords, or Facebook Ads, or any of the stuff. And a developer with the right idea is also liberated to be able to build just about anything they need — in that exact same way that musicians or video artists, or storytellers do. And that’s an amazing thing that’s happened.

      Sonal: Combining that with what David said about open source, it does create this sort of composability — build on top of each others’ building blocks. I mean, the best thing about TikTok is remix culture; like the fact that you can remix all these bits. And that’s exactly the same thing you’re talking about.

      You know this notion of developers want an audience, developers are a creative class — what does it mean for developers to become influencers more broadly within a company; with the question being, how to make developers more influencers across the company?

      Jeff: Well to me, really, that comes down to giving developers a voice. And an environment where you embrace experimentation — experimentation is the prerequisite to all innovation. You enable THAT as opposed to more hierarchical, top-down, highest-paid- person’s-ideas wins, and all that kind of stuff.

      David: You know Jeff, so many companies have not embraced the Ask Your Developer mindset. Sometimes what they do is they sort of find their way into the shallow end of the pool by… sort of having hackathons. And then magically, they find out that really good ideas come out of these hackathons. And hey wait maybe, maybe we should involve the developers earlier in that product-roadmap process.

      You know they have these good ideas, but they never get prioritized, they never get surfaced; like you said, they come from elsewhere in the organization, but maybe they shouldn’t. How do you think about hackathons? You know, when you’re talking to those business executives, how should they think about hackathons? And then how can they take that catalyst of sort of an event inside the organization and actually institutionalize that into their culture and workflow and process?

      Jeff: You know, I like hackathons, not necessarily because every hackathon results in the next giant innovation or whatever it is — you’re right, it often does end up proving the hypothesis that, oh wow, there are some things that we could do relatively quickly, that are very impactful, if we let our teams kind of go wild thinking about what are the things — but I like hackathons because they are a practice that actually encodes Ask Your Developer.

      ‘Cause if you think about it, inherent in a hackathon is this idea of letting developers essentially spend a period of time self-organizing and building the things that they think are interesting and important, and using that opportunity to prove out and to test out their ideas. In an ideal world, companies would operate more normally, in more hackathon-oriented ways — i.e., small teams working iteratively, and being agile, and being tasked with problems not solutions. And a hackathon is a way to simulate that, for a short period of time, and at small scale.

      Sonal: I mean, I hear what you’re saying, but I feel like hackathons are a bit performative. Because I’ve seen too many times, like a lot of companies do what you describe in the book as that “Silicon safari” effect, like animals in a cage; we must follow the same practices, and perform them essentially.

      David: I don’t think they’re performative. I think that they’re like, you have pressure building up in a system, and it’s like the steam valve — you reconfigure the machinery so that that steam valve doesn’t need the release.

      I don’t think there’s ever been an organization (at least that I’ve ever heard of) that’s done a hackathon. And been like wow, that was totally useless, we’re never going to do that again. They may not get that great new product that, you know, sends up their revenue for the next five years. But there’s always learnings — and those learnings are not just in the code that gets written, but in the processes that get created. So, I think hackathons are great. I would certainly not describe them as performative.

      Jeff: I will play the role of peacemaker here, because I think you’re both partially right. I think that, look, if you go into a hackathon saying okay I really am waiting for these folks to come up with the thing that’s going to save the company — it’s probably not really the right expectations to walk in with. So, to some extent, it is performative.

      But I think that the goal of the hackathon is not to solve the problem during the hackathon. The goal of the hackathon is actually to model what you want your organization to become; it’s like a rehearsal for really, the organizational structure and the way of operating during the regular course of business. And so I think that’s the role that hackathons play.

      I actually think a better way to structure it is, if you’re an executive at a company, create a two days a week, whatever you want to do; but go in with, hey, I care about this. You’re important, I’m committed to this. #1.

      #2: here is a list of the 10 biggest problems I hear from our customers; or here are the 10 biggest problems that we face as a company — and I’d love for you to be thinking about. NOW you’ve directed the energy, you’ve shared problems with those developers — and you’ve told them the stakes are high… I think that is a much more effective way to run a hackathon.

      Sonal: I love that. You’ve made peace.

      Developers’ unique workflow

      We’ve been talking about the developer mindset quite a bit. But we haven’t actually defined what is the developer way here: It’s not just a role and a function; like, it’s a mindset. And Jeff, have you seen in your work that these habits transfer across the org? You use the word “mindset” throughout your book; and David has used the word “way” throughout his work.

      I would love to hear your guys thoughts, kind of define what makes a developer.

      David: Look, I think developers in general — especially open-source developers — have mastered a whole bunch of working methodologies that end up just turning out to be great working methodologies not just for developers, but for anybody.

      So, that involves really having the tools to do asynchronous sort of communication and development; so in software development, that could be revision-control systems, things like GitHub, or GitLab, which allow people to collaborate. It can be ways of memorializing decisions: developers have change logs, they have issue tracking, they have pull requests — and so it’s often very easy to figure out how did this line of code get into the codebase; who signed off on that decision; who else reviewed it? And these are things that other organizations (outside the developer part of the organization), no one knows how the decisions get made; who made those decisions; when were they made; why were they made?

      And then, of course, there are power users of their own computing devices. And so, you know, developers often are much more keyboard-driven, they use shortcuts, they’re much more fast to operate. And we see these things bleeding into our world today; people now use emojis as shorthand. People are now using things like a command palette, and they’ve gone way beyond the way developers use command palettes; they’re now sort of bleeding into our normal daily life.

      But I do think there’s a lot to learn from the way developers organize, the way they communicate, the way that they memorialize decision making. And then, of course, the way they just use their computing tools as power users, because, you know, everyone’s effectively a digital native these days and becoming more and more of a power user.

      Sonal: How do you define it? Curious for your thoughts on this.

      Jeff: You know, I’m a little more hesitant to define like, the developer way. I struggle a little bit with saying, you know “here’s my definition of developers”, because there’s a lot of different ways to work.

      Now that said, a lot of developers do share a lot of common traits. Like when your work involves writing Boolean logic, a lot, if you tend to be drawn to that work, you probably tend to also want to have logical thinking in other areas of life. So I do tend to see engineers as being logical thinkers. And, you know it’s interesting, because like I, as a CEO (and a software developer), bring logic to a lot of the decisions, but also to a lot of my interactions with other team members.

      And I actually have noticed that it can be rather infuriating, actually — it’s one of things I’ve had to moderate as being a CEO, from being a developer — I’ve actually realized some of the ways in which the ways developers think, while they may be often right, they don’t necessarily serve you in interfacing with people who don’t think the same way.

      But I would say, if you’re a business executive, a few things to think about: One is, like many other arenas, where you have a lot of concentration required to do your job, *flow* is one of the most important things for developers — so the ability to immerse yourself in a problem, be able to kind of fit it all into the working memory of your brain, and then be able to get your work done is really important. That’s why developers are really sensitive to interruptions, to taps on the shoulders, or meetings, and things like that.

      And the other thing I would say is, if a developer is poking holes in the logic of your idea or your plan, they’re not being a jerk; it’s just the way they think. They’re processing whatever they’re hearing through the lens of how they think, and therefore, that’s the response you’re getting. And so, it’s maybe a way for folks to understand developers — and therefore be able to engage with them — is to think about the ways in which developers process information and make decisions.

      I like to propagate this idea that developers are creative problem solvers, and much bigger, more influential parts of the team, when they’re whole human beings. Not just like, you know, code monkeys.

      The importance of small teams

      Sonal: Jeff, you’ve alluded to this a few times — in fact, I thought this was one of the most interesting themes in your book — is, you asserted throughout it’s about small teams, it’s about small teams; it felt like a refrain.

      I’ve always heard the two-pizza rule for Amazon; I never heard the origin story until your book — and you describe having a dozen bagel team — so tell us a little bit about small teams, why they matter, how to grow them, how to make them work? I feel like the title of the book should also be, “Small Teams”!

      Jeff: Yes, “Ask Your Developer: Small Teams Are Right for You.” <Sonal laughs>

      So, back to, we were starting Twilio — at the very beginning, in this very small team that you are, you kind of do everything; everything from like having talked to customers that day, to handled support tickets, to writing code, to understanding the architecture of everything that’s going on. Like, you can hold the whole business in your head at the scale of several people.

      And as we started growing Twilio, one of the most momentous things that happened to me was I was talking to my friend — his name’s Dave Schappell (not the comedian, different Dave Chappelle) — he was actually the person who hired me at Amazon. And he had started at Amazon in, I think, ’97, so when the company was about 100 people. I joined in 2004, Amazon was about 5000 people. And Dave, he quit; that was my first week, he was like, “I’m sorry, I couldn’t tell you before. I’m out of here.” He went and started a company called TeachStreet, that was acqui-hired back into Amazon about seven years later. So, Dave found himself back at Amazon, but now the company was 75000 people.

      So, Dave saw Amazon at 100 people, 5000 people, and then again at 75000 people. And so, as I was starting to scale Twilio’s culture, and thinking about how we were going to structure ourselves, I called Dave and I said “Hey, Dave, can you compare and contrast Amazon at 100 people, Amazon at 5000 people, Amazon at 75000 people?” And he said, “Hunh, let me think about that for a second.” And he said, “You know what? It’s exactly the same. It’s the same bounce in people’s step, the same sense of urgency, the same intellect that everybody here has. It feels like the same company.”

      And to me, THAT is the outcome of the two-pizza team, as they call it at Amazon. Because as the company is growing, there’s a natural tendency for every company, as they get bigger, to slow down, to insert more bureaucracy, to create walls between customers, to create politics and things like that.

      And what small teams do is they keep a small group of people who are very tight, and focused on — what my definition is, the small team is defined by — a customer they’re serving, a problem they’re solving for that customer, and then metrics of success that say whether or not they’re succeeding. And there’s a lot of advantages here:

      First of all, on a very small team of say 10 people, there’s no room for a low performer to hide; on a team of 10 people, everyone’s got to carry their weight, and it’s obvious when somebody isn’t. The other thing that I think is interesting about small teams, is that people’s willingness to go along with decisions is proportional to how involved they were in that decision, and how close they are to the decision maker. And so if you’re on a small team of 10 people, and there’s a single-threaded leader to lead that team, then you want to push as many decisions as possible to that leader. And when you do, it’s likely that they’re going to be involved in that decision, and if the person’s managers would have made a decision that maybe they disagree with, they’re probably going to be more inclined to disagree and commit. Or, they’re more likely to be able to question it; hey, can you explain to me why you made this decision? Like you can’t do that when it’s someone five levels up; usually you don’t even know the person, or it’d be hard to get the meeting, or you’d be afraid to express yourself. And even when there’s disagreement, those disagreements get resolved. So instead of having this like us vs. them, you get this sense of: okay, you know we’re all on the same team here; you know let’s go, let’s do this.

      David: You know, one additional benefit of small teams that I’ve always observed is that there are some people that like to work on very small projects with rapid iteration, where they sort of have the dopamine rush of shipping a release and getting something done very quickly. There’s also other kinds of engineers that like really loooong, hard projects, that take months and months and have very little to show for it for a long period of time. And by having small teams, you can actually let people sort of work in an environment that works best for them. Like those people that want to close out a ticket to help win a deal, or save a renewal — like those people like to be on this fast, close-to-the-customer kind of teams; and there’s those infrastructure people.

      And, by having small teams, you allow people to end up gravitating towards the kind of work that ends up allowing them to work at their highest and best sort of potential. People can find where they fit in best, and I’ve always found that as an organization scales, to be a really, really valuable component.

      Jeff: The other interesting thing by the way, about the infrastructure people you mentioned — great engineers love building for the other builders, right? — but they’ve got to see it as I’m serving a customer with a mission and metrics.

      And so, even internally focused teams, it works the exact same way. And I think that’s one of the beautiful things about structuring yourself that way, is reminding everybody: Like, if a team exists and has no customer, internal or external, then, man, I’d wonder why they exist.

      Managing growth and team size

      Sonal: I want to probe into like, what happens when companies scale and grow, and, small teams can’t really stay small — You argue for a really interesting concept called “mitosis”, which obviously is borrowed from cells, that split as you grow. And I thought that was a really interesting idea.

      Jeff: Yes. So for us — I’ll give [an] example — our first product was Twilio Voice, the ability to make and receive phone calls with Twilio. And you know that was built by the founding team, we built it, we started hiring people, we grew. And suddenly the team that was working on that product became like 15 people. And we said okay, this is getting too big. If we believe in small teams, we need to split this. How are we going to do that?

      And so what you do is you take the problem domain, and you say okay, if I want to divide this problem domain in half — so I can have two teams instead of one big team — how would I do it? And there’s no one answer to it, but the best thing you can do is align the people, the technology, the code itself, and the customers. And when you can figure out how to actually divide the problem so that the customer, the technology, and the team can actually stay together, that’s the best way to do it.

      And so for like our voice product, we realized that the voice product really consisted of two things: One was the connectivity layer into the carriers of the world; and the second was all the programmable APIs that allowed you to do things with that connectivity. And so we divided those two teams. And initially, the code was completely intertwined, and it was like a complete mess. And we sat out and we said, okay, we need to decouple those two systems; and we need a technical leader for the connectivity side, we need a technical leader for the API side. And… over the course of about six months, we untangled the code bases, we untangled the teams. If we didn’t have a leader we needed for the next team, we would hire the leader. And after about six months, we were able to decouple the two, and take one very big team and turn it into two small teams again.

      And that’s basically the process that Twilio has used to grow from, you know the three engineers that we were when we founded the company, to now… several thousand engineers. We just keep doing this mitosis process.

      In the act of that, one of the key enablers of that is itself, APIs — and those APIs can be used internally, but they could also be exposed externally (if we wanted). And so we actually ended up doing that. We productized — we call it SIP trunking — that’s the connectivity layer, that is now its own product; and that product itself has undergone mitosis now many times, as well as our API layer, which is its own product.

      Do you throw away the notion of small team and say well, that’s only for the early stages, once it gets big, so be it. I think that’s exactly the wrong answer.

      Sonal: That’s fantastic, and I have to say, people really should read your book, because you say a lot more about the types of leaders that are needed, and I love that you have this line about — a phrase that you’ve coined — “The fallacy of better collaboration” — because that’s one of my pet peeves — where when you have too many small teams, how do you coordinate and collaborate? And it’s a wonderful, wonderful chapter.

      Lessons from Twilio’s IPO

      Last question. One thing that I’ve been dying to ask you, just super quick, which is, what would you say is your biggest personal evolution, pre- and post-IPO? That’s top of mind for a lot of people right now, so I’m very curious about that.

      Jeff: For me, it has been — the biggest evolution has been — really thinking more holistically about the intersection of product and go-to-market.

      You know, we went public, and we had about 12 sales people in the company. And we really loved our developer-first approach, our self-service model; developers sign up and start building. And… you know, we were very happy with that, and as such, like really had under-invested in sales. You know like 12 sales reps to manage a quarter billion in revenue, that’s an underinvestment, right?

      Sonal: Wow… yeah!

      Jeff: But what I came to realize was, empowering a developer to get started with Twilio is amazing. But once a company starts spending hundreds of thousands or millions of dollars, you can’t rely on a relationship with a developer to maintain that level of spend, because now you’ve got so many more stakeholders inside of the company.

      Developers want to do great work, but when the CFO is saying, “Hey, how come we’re spending this much on Twilio?” like “I don’t know, I… ” I mean, you know, that someone else’s job, right? And so we now we call it the “developer-first” approach, where, developers start the relationship, but then we build a mature relationship with many stakeholders inside the company — and that’s essentially what salespeople often do, is they understand the org chart of the company; and they understand who the stakeholders are; and they really build deep relationships with the customer (the customer, meaning the company, not just the individual.)

      You know so that’s probably I think one of the biggest things that I’ve come to you know, evolve, in my thinking is the holistic nature of what it takes to build a company.

      Sonal: It’s so funny, we have a whole series of podcasts called How to Go from a Technical to Product to Sales to Go to Market CEO, because it’s exactly the journey.

      That’s fantastic, Jeff Lawson, author, CEO of Twilio, and author of Ask Your Developer: How to Harness the Power of Software Developers and Win in the 21st Century. Thank you so much for joining!

      Jeff: Thank you, Sonal, it’s been a pleasure.

      • Jeff Lawson

      • David Ulevitch is a general partner at a16z where he invests in enterprise and SaaS companies. Prior to joining the firm, he was the founder and CEO of OpenDNS (acquired by Cisco).

      • Sonal Chokshi is the editor in chief as well as podcast network showrunner. Prior to joining a16z 2014 to build the editorial operation, Sonal was a senior editor at WIRED, and before that in content at Xerox PARC.

      The Machine that Made the Vaccine

      Stephane Bancel, Jorge Conde, and Hanne Winarsky

      In this special episode, which originally aired the day the FDA authorized the world’s first mRNA vaccine for emergency use, Moderna CEO Stephane Bancel tells the story of the machine that made the vaccine: the platform, the technology, and the moves behind the vaccine’s development.

      This episode of Bio Eats World takes us from a world of pipette and lab benches to a world of industrial robots making medicines: We used to grow our vaccines, now we can “print” them, getting them to patients faster and more efficiently than ever before. In conversation with a16z general partner Jorge Conde and Bio Eats World host Hanne Winarsky, Bancel describes the exact moment he realized they might actually be able to make a vaccine for COVID-19; what happened next to go from pathogen to design; how this new technology that uses mRNA works (in a chocolate mousse metaphor!), and what makes it different from “old” vaccines; and how to think about managing both innovation and speed in this world. Why is this such a fundamental shift in the world of drug development? And where will this technology go next?

      Show Notes

      • What happened when the SARS-CoV-2 virus was discovered, and Moderna’s response [1:44]
      • How Moderna’s vaccine was developed digitally [4:02]
      • Description of how mRNA drugs work [5:43] and the history of this technology [7:30]
      • The advantages of mRNA drugs [11:18]
      • How Moderna learned about the positive results from Phase 3 trials [14:41]
      • Why mRNA drugs can be produced more rapidly than other processes [19:06]
      • How mRNA technology might be used for other diseases, and where it is limited [22:18]
      • Details around Moderna’s manufacturing process [27:44]
      • How Moderna was founded, their original goals, and challenges faced [32:01]


      Hi, I’m Lauren.

      Hanne: And I’m Hanne, and this is “Bio Eats World”, our show where we talk about all the ways biology is technology.

      Lauren: This week, in place of “Journal club,” we have a very special episode featuring Stéphane Bancel, the CEO of Moderna, in conversation with you, Hanne, and a16z general partner Jorge Conde. And we’re talking about the COVID-19 vaccine, right?

      Hanne: Yep, that’s exactly right. The conversation is a really incredible dive into how they developed one of the world’s most awaited vaccines. Bancel describes everything from the moment he first realized they could make a COVID-19 vaccine with their technology to the day he heard the first data on how effective it was in humans. In this episode, which is airing just after the Vaccines and Related Biological Products Advisory Committee meeting makes its recommendation to the FDA, Stéphane tells the story of not just how the vaccine got made, but everything about the machine behind this vaccine — the fundamentally new platform and mRNA technology behind the vaccine’s development.

      Lauren: This vaccine is really one of the first medicines that is part of a bigger transformation from a world of pipettes and lab benches to a world of industrialized machines making medicines. We used to grow our vaccines, but now we can print them — getting them to patients faster and more efficiently than ever.

      Hanne: Bancel describes what it took to go from pathogen to design to clinical grade product; how mRNA works, in a chocolate mousse metaphor; and what makes it different from old vaccine technology — why exactly this is such a transformative shift in the world of drug development, and where this technology will go next.

      The discovery of SARS-CoV-2

      Jorge: Stéphane, sitting here now, in December of 2020, could you imagine a year ago that mRNA as a concept would be a household name?

      Stéphane: No, and we have a lot of things going on in vaccines, in cancer, in autoimmune disease, in cardiology, in viral genetic disease. But I had no idea that 2020 was going to look that way.

      Jorge: So, if we flash back to January of 2020, can you talk a little bit about the process that you went through to realize that you potentially had the technology that could be a solution for this emerging pandemic?

      Stéphane: Yes, so I’ve been working in infectious diseases all my career, and I’ve developed an eye for outbreaks. So, one of the things I do is I read the Wall Street Journal and the Financial Times every morning as I get up. And between Christmas and New Year of last year, I noticed an article saying that there is a new pathogen agent in China giving pneumonia-like symptoms. That’s all it says. And so, I sent an email to somebody working for Tony Fauci — Barney Graham, who we’ve been collaborating with for years designing several vaccines together. 

      And I said, “Barney, have you seen the new pathogen in China? What is it? Is it a bacteria or is it a virus?” And he replied to me a few hours later, and he says, “It’s not a bacteria. It seems to be a virus, but we don’t know which one yet.” And a day or two after, Barney sent me an email and said, “Hey, we learned from our contacts in China, it’s not flu, it’s not RSV. We don’t know what it is yet.” And then another day goes by and he says, “It’s a coronavirus, but it’s not SARS and it’s not MERS. It’s a new coronavirus. Within a day or two the sequence should be put online by the Chinese.”

      And so on January 11, the Chinese put the sequence online. And our team at Moderna used the sequence to design a vaccine. In parallel, Barney’s team does the same thing. And when they shared notes after around 48 hours, they had designed exactly the same vaccine.

      Creating Moderna’s vaccine digitally

      Jorge: A couple of things that are fascinating about this — number one, the fact that the digital copy of this virus came from China before the biological version reached our shores. That’s remarkable in and of itself, that we knew what we were dealing with, at least digitally, in a matter of days thanks to all of the advances with genomic sequencing technology. But the other remarkable advancement in technology here is what you just described — you were able to design a vaccine based on the digital version of the virus, also in a matter of days, is it?

      Stéphane: So, you’re right, Jorge. And this is the piece that I think most people in pharma don’t appreciate yet — the power of modern technology is, in 48 hours we designed and locked down the entire chemical structure of a vaccine.

      Hanne: Unbelievable.

      Stéphane: And we click “order” on the computer — so it all happened in silico, we never had access to a physical virus. And we designed the vaccine. And with, again, the two teams at NIH and Moderna, because we were so worried — to make a mistake in the vaccine design, as you can imagine.

      Jorge: Of course.

      Stéphane: So, we were super happy when the team literally compared notes after two days and they had exactly the same design for a vaccine, because it was an outbreak and we knew every day mattered. At the same time, we started to make clinical grade products to go into a Phase 1. And that is really remarkable — is the vaccine that is reviewed by the FDA on December 17, it’s exactly the same vaccine that our guys designed in January in silico. We never changed one atom. It’s exactly the same molecule.

      Hanne: So, it’s the same vaccine that took 48 hours to design.

      Stéphane: That’s going to help hundreds of millions of people next year, yeah.

      How mRNA vaccines work

      Hanne: Can we take a moment to just get really simple and talk about how you would define this messenger RNA technology, and what you wish the public understood about how mRNA works?

      Stéphane: Right. Yeah, so it’s a molecule that exists in every one of your cells that is basically the xerox copy of an instruction of your genome for one gene at a time to make protein in your cells. So the way I would describe it to my two young daughters is — think about DNA like the hard drive of life, where all the instructions of all your 22,000-ish genes are stored. And think about it a bit like this is a recipe book that your grandma gave to you before she passed away. All your favorite recipes, that’s the hard drive, that’s DNA. And when you want to make, let’s say, a chocolate mousse, if you go with your grandma’s precious book into the kitchen, you’re going to damage your book a lot. There’s going to be flour, and eggs, and sugar. And after a few times, you might not be able to read the recipe anymore. So what evolution has done, which is beautiful, is to protect the integrity of the instruction in the hard drive, in the book. When the cells want to make one protein, like let’s say insulin, what it does — it makes a copy of the instruction only of insulin in the book, like my example of chocolate mousse — a xerox copy — and takes it into the kitchen (i.e., the cell) to make for a little machine called the ribosome, that I describe [to] my kids as a little 3D printer that reads the message with the instruction of mRNA, and makes the protein by adding one amino acid at a time. So it’s a natural molecule that basically carries genetic information to make proteins.

      History of mRNA technology

      Hanne: But using mRNA as a tool in the way you’ve been doing it, that was not always an obvious approach. So can you talk a little bit about where that began, that idea, and what it first looked like?

      Stéphane: Yeah, so it’s actually very interesting. When mRNA and DNA were discovered, actually people in a lot of universities tried to make medicines out of mRNA because it was a very logical use of mRNA. Just copy nature — make a synthetic mRNA, inject it into animals before humans, and it should make a protein. Because of what was known about science at the time, including immunology — all the analytical tools that did not exist as part of manufacturing purity and so on — when they would inject mRNA in animals, animals would have flu-like symptoms — a fever, vomiting, diarrhea — because mRNA, if you remember, most viruses in life including COVID-19 is made of mRNA. And so through evolution, mammals have developed mechanisms to recognize foreign mRNAs. And of course, when you inject mRNA as an ID for a drug, that’ll be a foreign mRNA. And so actually people abandoned and just quit on trying to make mRNA as a drug. What happened in the 2010, 2011 timeframe, here in Boston, is you had a set of academics at Harvard and MIT who started to play with mRNA again because there [had] been some new discoveries made in the immune system — that they believed at the time that if you modified uridine, which is one of the four letters of mRNA, you can make an mRNA that’s immuno-silent.

      Jorge: In some ways, when you think about it, Moderna doesn’t make therapies, right? You make instructions that the cell uses to make its own therapies.

      Stéphane: Yeah, correct. We don’t give you the vaccine. We give you an instruction for the cells of healthy people, in that case — to read the instruction, to make one protein of a virus, to make it as well as if they had been infected by the real virus, to show it to the immune system so the immune system can make a neutralizing antibody and mature it. So that if, later, they get infected by the real virus, the immune system is ready to prevent the virus from [replicating] in their body and getting them sick. But what gets people sick in infectious disease is you have too many copies of a virus.

      Jorge: Yeah, and so I think this is a remarkable thing for a couple of reasons. What you’re essentially doing is you’ve looked at the virus’s, you know, genome, and you’ve said, “Okay, if I take certain pieces of code from this virus and encode them in mRNA and deliver them to human cells, I am basically giving the human cells the instructions to make pieces of the virus that the immune system will train itself on, recognize, and eventually neutralize.” And in this particular case, that target was the spike protein in the SARS-CoV-2 virus. Is that accurate?

      Stéphane: It sounds correct, Jorge. And the reason that mRNA, in my opinion, is so powerful is that you totally mimic to a human cell the natural biology of an infection without giving the virus at any time. We never give a virus to people — we give, as you said, a piece of the virus. In the case of corona, because it’s a pretty simple virus, we believed — and the clinical data have shown in the Phase 3 that we were correct — that one protein of a virus — a very important one, the spike protein — if you were able to get a high quality of a high quantity of neutralizing antibodies, you should be protecting people if they become infected with the real virus.

      Advantages of mRNA vaccines

      Hanne: Why is it better to have the cell mimic this natural process than in the old technology?

      Stéphane: And that’s a piece that is really unique, because when you think about it, when you get an infection by an mRNA virus in your body, what happens? The virus of mRNA gets in your cell, you use your own cell machinery to make the protein — to basically self replicate inside your cell — and then it escapes your cell. And this is what your immune system sees. And so if you think about it, the old technology of vaccines, where you’re making an E. coli cell or CHO cell — a protein that then you inject in a human, and that just circulates in your blood. That is not mimicking the natural biology. 

      In our case, the spike protein — we designed the vaccine, so it’s made inside the cell. So in a human cell, not an E. coli cell, and then we design it to be transmembraning — to stay attached to the cell, and to be presented to the immune system that basically backfalls, you know, in your blood, your body. And we see that thing sticking out of a cell — that is not [itself]. If you think about the 3D configuration of a B cell coming onto that protein, it is exactly like if it was a natural infection — which is why if you look at the data across the nine vaccines we put in the clinic, the antibody level is so high because it’s perfectly mimicking nature.

      Hanne: How did you know which protein and that one was enough? How did that process work?

      Stéphane: That’s a very good question. And, as Mr. Pasteur would say — and, of course, he has a big role in vaccinology — “only with a prepared mind.” So, one of the things we were doing with Dr. Fauci’s team for the last couple of years, is we [were] collaborating on studying viruses that could become outbreaks. None of us thought we [would] see over our lifetime a global pandemic. The last one we were all aware of as students of infectious disease is, of course, the Spanish flu. And so, one of the things where we got lucky is, we had been working for a few years together with Dr. Fauci’s team as part of that project for outbreak readiness on the MERS vaccine — the Middle East Respiratory Syndrome. 

      Which, if I had used those words a year ago, nobody would have known what I was talking about, but today everybody knows it’s another coronavirus. We wanted to provide to them mRNA for research grade — so, animal testing, antigen design, picking the protein that makes sense. Because mRNA is so easy to make once you industrialize it. We were able to send to NIH, to the team working on MERS, all the different vaccine designs they wanted to try in animals. They would vaccinate the animals and then they would challenge them by giving them a high-dose of a virus. The one that was most protective was always the spike protein. They tried a lot of combinations, but spike by itself was always the best.

      Jorge: And the theory, I assume, is because you’re essentially putting neutralizing antibodies around the spike and the spike is what the virus uses to get into cells in the first place.

      Stéphane: Correct. The full-length spike protein was always the best. Some companies went into a clinic with three, four, five candidates. And there were different hypotheses they were testing. We did not have to do that because we had tried it for a couple of years. We knew that with our mRNA, our best guess was going with the full-length spike protein.

      Success of Moderna’s vaccine in trial

      Jorge: At a very high-level, you are essentially printing these vaccines versus growing versions of a virus or a denatured virus. So you can design it, you can print it, and then you can, you know, obviously get this into people very quickly as a result. That is a remarkable part of this entire story that is probably somewhat underappreciated, that allowed you, and collectively us, to move so quickly. When did you know, Stéphane, that, all right, this is going to work, this is going to work for COVID?

      Stéphane: I had a very high belief that this should work since the beginning, so since January. Because this was the 10th vaccine we were working on. So it’s in the human data of a previous one. And in infectious disease — unlike in oncology, where the animal model tells you nothing — the infectious disease, if you look at a lot of data, there is extremely high translation from animals into humans. I saw MERS data before we started, of course, dosing in humans. So I knew the data in MERS looks great. So because we had done nine vaccines before, I knew it was going to look great in humans, which we learned all of this in May.

      Jorge: Can you describe, Stéphane, when you first saw the interim Phase 3 data and what your reaction was?

      Stéphane: So, it was a Sunday in November. I knew the independent NIH-led Safety Data Monitoring Board was going to meet at 10 a.m. on Sunday. And so I told my wife and my kids, I’m going to be a wreck the whole morning. I tried to pretend to work, but I was so distracted, I would check my email every two minutes, my phone every two minutes for a text message and so on. Maybe a bit before 1, I got a text from my team saying, “Hey, get on WebEx, we’re going to get the data.” There was not even a slide made. It was just somebody talking, literally reading to us the data.

      And so I learned about the close to 95% efficacy. It was already a big N and the p-value was very, very low. Very, very low. So this was real. And the piece that was almost the most exciting to me and my team was the severe case of disease, which there were, I think, eight or nine on the interim data. We have now 30 on the final analysis. And there were zero on the vaccine — they were all on placebo. And you think about what this means, when you connect those two data sets together, it means if you get our vaccine, you have a 95% chance of having zero symptoms if you get infected by the virus. You will not even know you are sick, you’ll just go live your normal life, zero symptoms. And in the 5% case, where you will get disease, it will be mild disease. You will get no severe disease. 

      And when you think about what has happened to our society — the elderly, people with high comorbidity, from hospitalization, when it gets bad [it] leads to death, and the total impact on the economy, the loss of jobs in so many industries, and so on — that whole cascade. If you could have a vaccine where most people, 95% get no symptoms, and the 5% who do get mild symptoms — never go and walk into a hospital — that will be a total game changer. So I listened to the data, then we talked to my team [for] a few minutes. No, I don’t think we were processing — and then I left my home office and I called my wife, she was in the house. And I told her, and I just started crying in the house.

      Hanne: I think that’s what it felt like for all of us hearing it too. It felt like, you know, normal life could return. It was the promise of something like that.

      Stéphane: We are losing, right now, 3,000 people in this country — I think it’s more than 10,000 people a day around the world. And it’s going to be a very tough winter. And that’s only the human toll, which is gigantic, but the piece I don’t think is talked about enough is the mental health toll happening to, you know, people at every age. All the young in, especially, you know, more disfavored communities where, you know, people are living in the small apartment, where Mom is trying to work. And kids trying — without a computer, without a good internet line, to learn remotely — the impact this will have in terms of equality. 

      And then, of course, so many industries have been totally destroyed. I mean, look, they are closing indoor dining again, which I think is the right thing to do. Because I think the most dangerous thing right now is to have dinner indoors. I have not walked into a restaurant indoors since March, and I won’t go until I’m vaccinated.

      Rapid manufacturing process

      Jorge: So, as amazing as I think the COVID vaccine story is, I think it’s also worth talking about the machine that made the vaccine — the technology platform that you have built over the course of 10 years that allowed you, in January of 2020, to say like, “Hey, we need to develop a COVID vaccine.” I remember coming to visit Moderna on Kendall Square, that first facility you had. And what was interesting about it is you walked in, it didn’t look like your typical biotech company. It was a row of machines, a row of printers, a row of robots. And that’s very different than what your traditional biotech company looks like. And it looked a lot more like an assembly line, in some ways. Where you can order something up and out the other end would come the mRNA medicine that you had ordered.

      Stéphane: Yes, and this goes back to this incredible property of mRNA, which I’m surprised that so many have missed — is that this is a disease and information-carrying molecule that you can industrialize. When you are in an analog business — which is what I think all pharma and all biotech is, in my book — it’s because every molecule is a different chemical entity, you cannot industrialize the making of a lot of it at the research grade. You have to literally have chemists, and pipettes, and so on. You know, doing like we all did in chemistry class, writing the synthetic route to get to a molecule that they want to do the biological effect they want. 

      And then they have to design that chemical equation, and then all the pipettes and test tubes to do that. And when it’s another molecule, they have to invent another synthetic route. So it’s very — an analog world where you invent everything once at a time for one product. Because if every product is different, you have to re-optimize every time, and sometimes it’s very complicated because of very complex biological systems. So sometimes it will take you 6 months, 12 months, 18 months to get ready from preclinical data to be making clinical grade product that you need to file to FDA so that they give you the green light to go into testing this in humans. It’s highly regulated — as it should be processed to protect people’s safety. But in our case, it’s always the same thing, because mRNA is always made of [the] four same letters — the four letters of life, like zeros and ones in software. It’s the same manufacturing process. 

      This is like software or LEGO, this is an engineering problem. It’s an engineering technology, it’s a platform. The only difference between all Zika vaccines, or all CMV vaccines, and the COVID vaccine — it’s only the order of the letter; the zeroes and ones  of life. The manufacturing process is the same, the equipment is the same, with the same operators. It’s the same thing. And so this is why we could go so fast. It took us 60 days to go from a sequence of a virus presented by the Chinese to dosing a human. The first SARS — SARS-CoV-, or as it was known before SARS — it took the NIH 20 months <Mmhmm.> to go from sequence to starting the Phase 1 study. So, you went from 20 months to 2 months.

      Possible future uses for mRNA drugs

      Jorge: Which is remarkable. Are we in the plug-and-play future for vaccines?

      Stéphane: Oh, 100%. We’re going after making a seasonal flu vaccine — because, as we all know, still 10,000 Americans die every year, on average, of seasonal flu. We believe that we should be able to make a big dent [on] flu. And today we have six vaccines in development. We’re going to have many more soon, because for 10 years, you know, Jorge, we hoped that mRNA vaccines were going to work. We believed scientifically they were going to work — but until you have a Phase 3, randomized, placebo-controlled study where you test for the prevention of disease, you don’t know. Now we know.

      Hanne: Are there limits right now to how sophisticated these instructions can get, or can we essentially give them as sophisticated instructions as the human body is capable of?

      Stéphane: It’s — when the mechanism of a disease is not well understood. So we spoke about vaccines, and we said, “Look, coronavirus,” as I said, “is actually a simple virus.” We, as a society, got lucky. Think about HIV. HIV [was] discovered 40 years ago. There is still to this day no approved vaccine against HIV. Think about the awful world we would be in right now if Dr. Fauci had been standing on the presidential podium back in spring, and told them, “Folks, I’m sorry to tell you, but this is an awfully complex virus. We have no idea when we might have a vaccine.” Think about the state of mind we would all be in now. The biology of viral genetic disease is very well understood. Why? Because kids got two biogenetic information from their parents that they cannot make a correct protein, and that is what causes their disease. They have a wrong instruction in their DNA. 

      You can give them an mRNA from our technology, coming in their cells with the right instructions — then they will have the right protein and they won’t get sick. If you think about cancer, on the other hand of the spectrum, or Alzheimer’s now, if the disease mechanism is not understood, we cannot drug it easily. We can try things, of course. We could make an mRNA behind that hypothesis, go try it in a clinic — but a lot of things will fail because we are guessing. And so the piece where I think we have an incredible tailwind — basically overlaps doing academic biology work around the world are helping us. Because if tomorrow there is a paper published by our lab in the U.S., or in China, or anywhere in the world that says protein X, Y, Z is the root cause of that disease, or those five proteins in this ratio are the root cause of that disease, then we can literally turn on the computer and, you know, design a drug to go test that hypothesis in an animal.

      Jorge: Basically, the power of this approach works when you know what you want to make and then you just need to deliver the instructions to make that. Where it doesn’t work as well is when you’re not quite sure what it is that you need to make.

      Stéphane: This is basically biology complexity or biology risk. The other dimension for us is the ability to deliver the mRNA in the right cell. We actually have become a “delivery of nucleic acid” company. We realized that what would allow us to maximize the impact we could have on disease, or helping as many people as we can over the next 5, 10, 20 years, is the ability to bring up mRNA to different cell types. So a good example today is, if you say, look, there is this university that published the mechanism of Alzheimer’s disease. If it happens in the brain and we don’t know how to bring mRNA [to] the brain safely, we cannot drug it. So the biology will be understood, but the delivery technology will not be there.

      An example where we’re making a lot of progress right now is the lung. <Mmhmm.> We have been working with Vertex around how to deliver mRNA via an aerosol via your mouth into your lung, because they know the biology very well. And we work together to develop a delivery system to bring mRNA safely into your lungs, and to bring enough mRNA at a safe dose to get the biological effect. And we’re getting very close now. Once we can prove in the clinic that that delivery system works, then the next morning you can make any other drug you want that you need to get into the lung, because it’s getting another set of zeros and ones coded differently, with the same delivery system into the lung. And that’s the power of the technology — which is why with vaccines we’re able to go so fast.

      Jorge: Yeah, the instructions have gotten so sophisticated over time that now the next sort of horizon is, you’ve got to get the vehicle for delivery equally sophisticated.

      Stéphane: We’re adding vertical, after vertical, after vertical — then we bring mRNA into a new cell type. So the vaccine is one vertical. Getting mRNA into a tumor is another vertical. We have a very cool drug, where we inject mRNA in people’s hearts after a heart attack — and here we code for a protein called VEGF, for the biology geeks on the podcast, V-E-G-F. That is a protein that we all have the instruction in our DNA, which basically tells your body to make a new blood vessel.

      Hanne: Amazing.

      Stéphane: You use that protein every time you cut yourself.

      Using robotics to manufacture drugs

      Hanne: Stéphane, you’ve mentioned, you know, kind of the fast design of the vaccine, and then you mentioned even robots printing medicines. Can we get your version of what that machine assembly line looks like?

      Stéphane: So, the robotics farm we have in our factory is basically just an assembly of robots that get instruction coming directly from computers. There’s no human interaction. And basically, you start from a piece of DNA. That is basically your template. You put that in a reactor with water. There is no cell — it’s a cell-free manufacturing process, which is why it’s so fast. And you put enzymes. And basically, what the enzymes do, they attach to the DNA, and they read the DNA template. And they quickly tell pieces of nucleic acid — i.e, the zeros and ones, the four letters of life — they bind them next to each other to make an mRNA molecule. Then the robot goes to the next step, which is you add a cap. 

      Think about it like the nose of a molecule, that you add again with another enzyme. Then what you do, you purify the mRNA. So basically, you pick the mRNA from all that water, enzyme, and nucleotide, nucleic acid, and so on. And then when you have a pure mRNA molecule, after purification, you mix it with a lipid, i.e, fat. And that fat basically goes around and packages, like in a little bowl, the mRNA to protect the mRNA in your blood, and to get the mRNA inside your cells. When it’s inside your cells, the lipid — the fat — falls apart, the mRNA is released inside the cell, and the little ribosome — the little 3D printer of your cell — is going to read that message, make the protein on demand, and here you go — the patient, the human is making his or her own medicine.

      Jorge: I remember from the earliest days you were obsessed with the operations. You were obsessed with turnaround time, with throughput, with, you know, cost per output. And the benefit of that approach is that it obviously just compounded over time. The benefit of the technology, as you’re describing it, is that you have a machine that prints the instructions that go into the cell — that uses the cell’s machine to make the medicine, or to make the vaccine. And that’s this incredibly powerful paradigm, you know, to taking therapeutics or vaccines from being very bespoke efforts to being truly industrialized, designed efforts.

      Stéphane: That’s what is really so powerful is that the whole drug process is all about information. The piece that is remarkable is you have this very modular technology, because what happens in our cells is actually extremely logical. We start from the sequence information of a virus, like in the case of a COVID vaccine, or we use the human genome. We put [it] into a technology genetic-based cassette, and then you click “order” on the computer and you go again. And that’s the vision I always had since day one. And a lot even of my scientists thought I was crazy, because this industrialized, engineer-driven approach to drug discovery has never happened [before].

      Hanne: So, Stéphane, you’ve described this process which is, you know, much more efficient, industrialized in nature, incredibly fast compared to the old process. Is there a world in which that gets even faster? Are there other things — you know, other increases in technology that would speed this up even more?

      Stéphane: Yes, so it took us 42 days to go from sequence to shipping the human grade vials to Dr. Fauci’s team. The big bottleneck is sterility testing — a very important quality control test that is done for any injectable pharmaceutical to make sure that there’s no bacteria in the product. That test takes two weeks, because what basically you do, you take a sample of your vaccine and you wait enough time. If there’s only one copy of a bacteria, by that time you have enough multiplication of bacteria, through the detection of the assay of a test that you will see it, you will not miss it. It’s very important for people’s safety. Well, if there was a technology developed where you could do sterility testing in one day with high sensitivity, then you could take our process down to two weeks.

      The history of Moderna and its approach

      Jorge: So, we’ve talked about the vaccine. We’ve talked about the machine that made the vaccine. I’d love to take a second to talk about the company that built the machine. So from the moment that you started this company, you took a very different approach. And you’ve described it as having an engineer’s mindset. Can you talk a little bit about what you did, and how you thought about the early company build?

      Stéphane: I had never built a company in hypergrowth. You know, I worked at Eli Lilly, I ran bioMérieux, which is a big diagnostic company. But I have never built myself a company building very, very quickly. We decided to do something very atypical, because most biotech companies are one-drug company at a time. What was very clear to us, because mRNA is an information molecule, is it made no scientific sense that this will be a one-drug company. It will be either zero, because we run out of money before we can safely get the drug approved, or it’ll be a company with thousands and thousands and thousands of drugs because of the platform.

      And so, once we realized that, in the first hours of talking about Moderna, we started to become very worried and paranoid about, “Geez, we don’t know what we don’t know about this technology because it’s new. It has never been approved.” And, “Geez, if we pick one drug, if we are wrong and it doesn’t work in the clinic, everybody will believe what people have believed for 50-plus years” — which is, mRNA will never be a drug. And we most probably are going to go bankrupt. But if mRNA could’ve worked, we will have failed society. Because if we find a way to make this work, this will [mean] thousands and thousands of drugs that are undoable using existing technology — like the VEGF in hearts — and we will shortchange societies, shortchange patients. And that was just unbearable.

      And so we spent a lot of time thinking about, okay, what are all the things that could make us fail? We ended up zooming [in] on four risks that we say — if we can manage and reduce those risks, we will have [the]  best chance to be the best version of Moderna. Those risks — we’ve talked about very publicly, especially when we went public. It’s technology risk around the mRNA technology. So, of course, if you do a new technology you don’t know what you don’t know. There’s going to be a lot of risk there of things not working as you expect. Two is the biology risk. You can have incredible risk that your scientific hypothesis on the biology is incorrect and the drug will fail — not because the technology wasn’t working, but because the scientific hypothesis on the biology is incorrect. Then there was going to be a lot of execution risk. And then, of course, financing risk. Because we said, like, you know, asset managers build a portfolio — we said “Picking one drug is crazy, it’s like buying only one stock.” And so we said, “Let’s build a full portfolio of drugs.”

      And after many, many months of discussion, we designed basically a pipeline of 20 drugs that we said we’re going to take all those drugs in parallel to the clinic, so there would not be a binary event that the company makes it or not on one drug. So we diversified the technology risk around six different technology applications, from vaccines, to [a] drug in the heart, to a drug in the liver for a genetic disease. And then for every application, we took several drugs to diversify the biology risk. And we launched that crazy experiment with, you know, 17 drugs in the clinic so far — which was going to create incredible execution risk because it’s harder to do 17 at the same time than 1. And incredible financing risk because we needed a lot of capital. But we traded those risks with our eyes wide open, because the other risk could kill us with much higher probability — the technology and the biology risk.

      Jorge: It’s very difficult in this industry to take that balance, platform versus programs. And, you know, what tends to be the case very quickly is most companies when they have to choose where to put an incremental dollar, or an incremental head, they put it on the programs because those are the golden eggs and they want to move those forwards to create value inflection. And as a result, the platform ends up getting starved. <Yes.> You started the other way around. You actually fed the platform, and you fed the goose, and then let the goose lay its eggs.

      Stéphane: Yeah, exactly. The goose is more valuable than any egg. If you really believe you have a goose that’s going to be making thousands and thousands and thousands of eggs, you don’t want to kill the goose on the first or second egg.

      Jorge: Although most geese are not that fertile in biotech. Yours… <crosstalk, laughter>

      Stéphane: And that’s why I told you both that I was not interested [in going] public early because the capital markets were going to force me to not invest in the goose. Because biotech firms like to bet on eggs, not on [the] goose, because there has not been a lot of geese before in this industry. So we’re not used to it.

      Jorge: I mean the record will show that you did a lot of things right. As you built the company over the last 10 years, can you talk a little bit about the things you did wrong, that if you could get them back you would do it over?

      Stéphane: Well, [the] easiest one, given the COVID situation is — it took us three years to start working on vaccines. So think about how the world would be different and Moderna would be different if we started working on vaccines from day one. We might have been able to go even faster for COVID. So that’s a thing I regret, and that’s on me. I made quite a lot of mistakes hiring people, because I underestimated how intense our company is because I live it every day. I thought initially that it was obvious that this is a small company fighting for its life, so people are going to work hard. It’s brand-new, cutting-edge science, so it’s going to be complicated because every other thing is not going to work. 

      So, being able to manage uncertainty — people having a lot of grit. Collaboration, because making a drug is a team sport. A drug is a system of so many capabilities — the biologists, the [toxicology] people, the chemists, the engineers to make the drug. And a lot of times, people coming from big pharma are used to working in silos. And people who come from academia don’t know how to develop drugs. It’s a system. And like any system, you get the best outcome if you really optimize the system working together.

      Jorge: So, last question I would ask you — what advice would you give to the engineer that wants to get into biotech?

      Stéphane: So, first he needs to learn a bit about biology. I mean, I had a chance, as I spent my entire career in biology, so I’ve learned a lot on the go — I’ve learned a lot by reading. I’m a curious guy, so I read a lot. You can get biology books and learn. And I think it’s understanding enough of biology so that you can be part of a conversation, so that you can have an impact on decisions and scientific choices that happen. And then you can go from there.

      Hanne: That’s wonderful. Thank you so much for joining us on “Bio Eats World,” Stéphane. We’re so grateful for your time.

      Thanks so much for joining us on “Bio Eats World.” If you’d like to hear more about all the ways biology is technology, please go subscribe to the a16z bio newsletter at a16z.com/newsletter, and of course, subscribe to “Bio Eats World” anywhere you listen to podcasts.

      • Stephane Bancel

      • Jorge Conde is a general partner at Andreessen Horowitz where he invests in companies at the cross-section of biology, computer science, engineering. Before a16z bio, he was CSO at Syros, cofounded Knome, & more.

      • Hanne Winarsky

      The Cost Disease in Healthcare

      Marc Andreessen and Vijay Pande

      How come things like healthcare, education, and housing get more and more expensive, but things like socks, shoes, and electronics all get cheaper and cheaper? In this episode of Bio Eats World, a16z founder and internet pioneer Marc Andreesen, and general partner Vijay Pande, discuss the lesser known law of economics that explains why healthcare, education and housing is so damn expensive, and getting worse.

      What’s really at heart is tech’s ability to transform (expensive) services into (affordable) goods: think of the cost of a live string quartet, versus a streamed recorded track; or the cost of a custom-made shoe, versus a factory-made one. Until now, using tech to similarly transform services into goods in healthcare has seemed like an impossible dream — how would you do this for, say, the service of doctors providing care? But in this wide ranging conversation all about technology and society across all industries, Andreessen and Pande talk about the massive new gains recent technologies have begun to make this seem within reach, from eye surgery in malls to using AI in processing medical claims. Is there a future in which what doctors are doing today feels analogous to farmers hand plowing fields 300 years ago? And what would the role of that doctor of the future be?

      Show Notes

      • How Baumol’s cost disease is distorting pricing in healthcare and education [1:56]
      • How technology could reduce healthcare costs [5:52], just as it has in the past with other goods and services [10:41], and LASIK as an example of market-driven healthcare [13:33]
      • The role of individual behavior in chronic health conditions [17:20], and ideas for how this can be managed [19:56]
      • Using apps and wearable devices to drive positive behavior change [25:15]


      Lauren: Hi, I’m Lauren.

      Hanne: And I’m Hanne. And this is our show, “Bio Eats World,” where we talk about all the ways our ability to engineer biology and re-engineer healthcare are transforming our future.

      Lauren: So, Hanne, this episode is called “The Cost Disease in Healthcare.” What disease are we talking about?

      Hanne: It’s actually a reference to what’s called Baumol’s cost disease, or the Baumol effect, which is a phenomenon first described by an economist named William Baumol in the 1960s. In short, the Baumol effect is when there’s a rise in wages and jobs and industries that then haven’t had the same rise in productivity.

      Lauren: Okay. But what does that really mean, and what does it have to do with healthcare?

      Hanne: That’s exactly what this episode is about. a16z founder and internet pioneer Marc Andreessen and General Partner Vijay Pande discuss the economic forces that make some things like healthcare, education, and housing get more and more expensive but things like socks, shoes, and electronics all get cheaper and cheaper.

      In this wide-ranging conversation about how society and different industries work and what that means for consumers, Marc and Vijay talk about tech’s ability to transform expensive services into affordable goods. Think of the cost of a live string quartet versus a streamed recorded track, or the cost of a custom-made shoe versus a factory-made one. But until now, using tech to similarly transform services into goods in healthcare has seemed like an impossible dream.

      How would you do this for, say, the service of doctors providing care? Marc and Vijay talk about the massive gains in new recent technologies that have begun to make this seem within reach, from laser eye surgery in storefronts and malls to using AI in processing medical claims.

      Is there a future in which what doctors are doing today feels analogous to farmers hand plowing fields 300 years ago, and what would the role of that doctor of the future look like? Take it away, Marc and Vijay.

      Baulmol’s cost disease

      Vijay: So maybe, you know, the place to kick this off would be to talk about what is Baumol’s cost disease and why it’s so important. I think maybe the Twitter version of it is, how come things like healthcare and education and construction exponentially increase in cost, whereas socks from Walmart or many other goods, especially anything electronic, decreases exponentially. How could that be? I mean, we’re living in this world where things magically get cheaper, but that college education or healthcare just is becoming this massive challenge for us as a nation.

      Marc: Yeah. So, the way that you measure the impact of technological change in society is through what economists refer to as productivity growth. It’s the process of figuring out how to make more output with less input, right. And so, normally, we kind of expect the world to work this way, which is, every year over the last 300 years with a couple of exceptions, most industries got a little bit better at making things and costing a little bit less, and that led to this huge rise in living standards.

      Agriculture is kind of the classic case where food is really cheap as a consequence of a lot of technological leverage applied to the challenge of growing food, and we just generate a lot more calories of food for a lot less money now than we did 100 years ago or 300 years ago.

      The problem or the challenge is that different sectors of the economy have different rates of productivity growth, basically, depending on their idiosyncrasies and then depending on the extent to which technology is empowered or allowed to actually have its effect on things.

      And so you see these industries like consumer electronics and media and food and clothing in which you’ve got this spectacular productivity growth and then correspondingly these spectacular price declines over time. And then you have these other economic sectors — education and healthcare and housing as three in which the price curves are going in the wrong direction.

      The cost of a college degree gets more expensive every year. The cost of heart surgery gets more expensive every year, which is going backward, basically neutral, and maybe even negative productivity growth. Like, they might be getting worse over time.

      Then you basically got this problem where you’ve got certain industries that are racing ahead in productivity growth, and so, those workers are kind of becoming super technologically empowered to produce a lot more with less input. And so, those workers are actually getting paid a lot more because they just got so much technological leverage to the work that they do.

      Think about, for example, the producer of a TV show or something like the level of kind of power that you have with a modern computer to like produce a TV show. It’s leaps and bounds beyond what you would’ve had if you were literally cutting, you know, splicing film, you know, by hand with the scissors and tape, which is how things used to work.

      And so you’ve got these industries in which productivity is growing very fast, prices are declining, and wages are exploding. And then you have these other industries like education and housing and healthcare in which that’s not happening, but the problem is workers can actually migrate from sector to sector. If I’m not excited enough about having a job as, like, I don’t know, a film editor or something, like, I can go to nursing school and I can become a nurse. And now I’ve migrated out of the media industry, and I’ve migrated into the nursing sector.

      And then the problem is wages get set across these industries, and so you basically have industries with neutral and negative productivity growth that are now setting wages as if they had positive productivity growth, which they don’t. And then the result of that is just this explosion of price in those kinds of negative productivity sectors. It’s just horrible for consumers of healthcare, education, and housing because the same stuff gets basically more expensive every year without getting any better and maybe getting worse.

      Oh, and then, the other big problem is, there’s no reason why this would ever stop. The way I would crystalize this whole thing is — because of rapid productivity growth in consumer electronics, a 100-inch big-screen TV that goes on your wall, in your house, and you can watch every movie ever made for $10 a month. The price of that TV is going to drop to, like, $100. That is, like, quite literally what’s happening.

      Correspondingly, the price of a high-end private four-year university degree has leaped up dramatically over the last 20 years. It’s now in the order of, you know, $75,000 a year. So it’s like $300,000 for a degree. It’s not going to be that long before a four-year college degree costs a million dollars.

      And so you’re going to have a $100 television set that covers your whole wall, and you’re going to spend a million dollars getting a college degree. And that’s just crazy. It’s just, like, such a horribly bad outcome, and yet there’s something in the structure of how these markets work that prevents us from kind of speaking openly about the trends that result in this.

      Technology and AI in healthcare

      Vijay: Yeah. I think, Marc, one key point that you laid out there was that this is very much the cost of labor and that there’s a sort of specialized labor. And in many of these industries that we’re talking about — healthcare, education — this is an apprenticeship, where you have to spend many years to be able to develop skills that are handed down from one person to another. Very, kind of, pre-Industrial Revolution kind of behavior. Whereas when you talk about goods, goods are made in factories that are completely automated, and that technology can be applied there to make them 10% better a year, and that leads to exponential performance over time.

      And one of the key ideas that I was curious to hash out with you is what we’re seeing in AI. What we’re starting to see is that AI is turning what used to be something that had to be done by a service into something that can be thought of as a good. That instead of a person training in an apprenticeship-like way to do something, the machine learns these things. You can make copies trivially. You can get now the advantage of Moore’s Law, and this almost alchemical magical transformation seems to be one part of a potential solution to addressing Baumol’s cost disease. I’m curious how you see, at least, that part of it?

      Marc: Yeah. So this actually goes to an example that Baumol used when he wrote the book, kind of on this topic. And so he used the hypothetical example of, like, a string quartet, right. There’s two ways to experience a string quartet, right, in your house. One way is the old way, which you can actually hire four musicians to come and set up in your house and play Beethoven quartets, and it’s going to sound great. And by the way, people still do this. You do this for, like, weddings, right, this is still a thing.

      The other way to experience a string quartet in your house is electronic playback, a recording. And what’s the cost of a string quartet recording to playback in your house today? You know, it’s basically zero. If you just chart the price of getting four musicians to come play at your house from, like, 200 years ago to 100 years ago, to 50 years ago, to 20 years ago, to now, that price has exploded. The in-person version has gotten, like, wildly more expensive, right, because of Baumol’s cost disease, because those musicians actually work for a living, and they have other career options.

      And then, exactly to your point, in AI, what’s the simplest form of AI? It’s a computer literally listening to what’s happening and playing it back. And it turns out that costs nothing. An enormous amount of progress in the modern economy is that, right. You also benefit from that, by the way, every time you buy a loaf of bread. Our ancestors were not buying loaves of bread carefully pre-sliced off the shelf. Our ancestors, to the extent that they were able to get access to the core ingredients in the first place, were, you know, making bread by hand.

      Vijay: Yeah. Baker as a service.

      Marc: Yeah. Exactly. This is actually the big lever that increases living standards. Exactly to your point also, it has been hamstrung by the fact that historically computers have only been able to do so much. Machines have only been able to do so much. And now we have these sort of much more flexible technologies kind of gathered under this term of AI that at least in theory give us the opportunity to now revisit a lot of our assumptions about what should be a product and what should be a service.

      Vijay: Yeah. One of the fun things that we’re seeing is AI is nibbling in with the easy, mundane things that are annoying for people to do that they have to be trained to do, but then, that training that goes to people can now be done to the AI, and the AI can be trained once and then scale and actually learn from everyone else’s mistakes.

      And so what we’re seeing, as initial go-to-markets in healthcare are in areas of billing or simple types of diagnoses or triage — areas where this isn’t trying to make some superhuman genius, which may in time come, but I think the first go-to-market is just taking the things that are just boring and reproducible that are just expensive because of the human power involved, not even necessarily because you need a super genius. And that’s something that we’re seeing right now.

      Marc: There’s a famous story of Alan Turing where he was working on inventing the computer in the early 40s, and he was hanging out at Bell Labs in New Jersey with his friend Claude Shannon who’s the inventor of information theory. The two of them were having this heated lunch discussion at the AT&T headquarters building in New Jersey with all the top researchers and AT&T executives kind of sitting around nearby about basically this concept of AI. Like, what would it mean for computers to actually be intelligent to actually have brains.

      And they were debating back and forth, and finally, Turing got frustrated, and he stood up and yelled at Claude Shannon. He said, “Look. I’m not talking about turning a computer into a super genius. I’m just talking about building a mediocre mind, like the preside